Muse：自 ChatGPT 以来最大的 AI 发布

现在发生的事情意义重大。这是自 ChatGPT 以来最大的发布。它叫 Muse。

What's happening right now is a big deal. This is the biggest launch since ChatGPT. It's called Muse.

Muse。它让科技界沸腾。22 天 500 万次下载，超越 ChatGPT。而这一切是他打造的。

Muse. It has the tech world buzzing. 5 million downloads in 22 days, surpassing ChatGPT. And he built it.

大多数人用 AI 来回答问题。基本上就是美化的谷歌搜索。这是完全不同的东西。它是一个为日常生活打造的 AI 智能体。

Most people are using AI to answer questions. Basically, glorified Google search. This is something totally different. It's an AI agent built for everyday life.

我们相信 Muse 会帮你赚钱。有人让 Muse 帮忙规划与妻子的周末旅行。它能打电话。它能为你完成任务，但要让它发挥作用，你必须信任它。首先是信任它处理你的个人信息，然后是信任它会做你真正想让它做的事，最后是信任它不会失控。而这一切发生的时候，最重要的 AI 领袖们正在警告我们，AI 正在失控。

We believe that Muse will make you money. One guy had Muse help plan a weekend away with his wife. It can make phone calls. It can do tasks for you, but for it to work, you have to trust it. First with your personal information, then to do what you actually want it to do, and then that it's not going to well get out of control. All at a time when the most important AI leaders are warning us that AI is getting out of control.

我敦促你敦促我停下来。如果你理解 Muse 现在正在发生什么，你就会理解 AI 领域正在发生的最大事件。那么，它到底能做什么？我们什么时候该信任它？我们需要做什么来确保这件事走向正确？

And I urge you to urge me to stop. If you understand what's happening right now with Muse, you'll understand the biggest things happening in AI. So, what can it actually do? When should we trust it? And what do we need to do to make sure this goes right?

我们正处于某种疯狂巅峰的时刻。为什么这些亿万富翁都痴迷于餐厅预订？我们有一个共同的利益，就是确保这项技术不会走向可怕的错误。那是在某个私人房间里发生的事吗？某些个人恩怨竟然在决定他们如何开发这项极其重要的技术，这在我看来简直疯狂。太棒了。太好了。好，我们开始吧。欢迎。

And we're at the moment of like peak insanity. Why are all these billionaires obsessed with restaurant reservations? We have a shared interest in making sure this technology does not go horribly wrong. Is that something that's happening in some private room somewhere? The degree to which certain personal animosities are actually defining how they're developing this really important technology is insane to me. Amazing. Great. All right, let's do this. Welcome.

你最近怎么样？

How have you been?

我很好。真的很好。你怎么样？这真是

I'm good. I'm really good. How you been? This is

有点疯狂。是的。

It's been kind of insane. Yeah.

是的。嗯，感觉怎么样？

Yeah. Um what does it feel like?

嗯，我们没想到它会这么火。所以，我为整个团队和所有为此努力的人感到非常棒，因为它真的凝聚在一起，成为了一个现象。这就是我兴奋地想聊的。

Well, we didn't expect that it would be such a hit. So, I feel amazing for like the whole team and everyone who like works so hard on it cuz it's like it's really come together and become a thing. So, that's what I'm excited to talk about.

我知道你已经很了解这个节目了。但这不是传统采访。我不会问财务数据。我不会问管理风格。你刚刚发布了史上最受欢迎的消费工具之一。所以我唯一想知道的是，你正朝着那个工具的什么未来努力，以及这个非常乐观、非常想参与确保那个未来走向正确的观众，他们怎么能感觉自己参与了正在做出的决定？

I know you know all about the show already. Y but this isn't like a traditional interview. Like I'm not going to ask about financials. I'm not going to ask about management style. You just launched one of the most popular consumer tools of all time. And so the only thing that I want to know is what's the future that you're working toward with that tool and how can this audience that's really optimistic and really wants to be part of making sure that that future go right. How can they feel like they're part of the decisions that are being made?

是的。所有这些。这就是我们做的。

Yep. All of that. That's what we do.

我想和你聊的原因之一是我一直很欣赏你的直率。我真的认为人们需要关于这件事的直白对话。我觉得他们感到困惑，我想弄清楚到底发生了什么。你构建了什么？人们如何使用它？如何不使用它？真正的挑战是什么？我认为另一个有趣的事情是，鉴于 Muse 变得如此受欢迎，我想用它作为谈论 AI 其他事情的方式。

One of the reasons I wanted to talk to you was I've always appreciated that you're really blunt. Like I I honestly think that people need straight talk about this. I think that they feel confused and I want to get at what's really happening. What have you built? How are people using it? How are they not? What are the real challenges? I think one other interesting thing is given how popular Muse has become, I want to use it as a way to talk about other things in AI.

是的。

Yep.

太棒了。好，我们开始吧。Muse 是自 ChatGPT 以来增长最快的消费工具，我认为比 ChatGPT 还快，感觉这可能是我们从聊天机器人转向智能体的时刻，但已经有很多其他流行的智能体，主要是在代码领域，我很好奇你为什么认为 Muse 现在会发生这种情况，为什么 Muse 会这样爆发？

Awesome. Okay, let's get into it. Muse is the fastest growing consumer tool since ChatGPT faster than ChatGPT I think and it feels like this is the moment when we're moving maybe from chatbots to agents but there have been a lot of other popular agents mostly in code and I'm curious why do you think this is happening for Muse right now why is Muse exploding like this

我认为今年到目前为止，AI 的谜团之一就是，智能体一直在为工程师、软件工程师和程序员发生，而世界上其他人并没有体验到它，不知道它是什么，不理解到底发生了什么。所以今年大部分时间，你几乎有两种对话，做代码的人能看到 AI 变得多么强大、多么有用，他们会说：“天哪，这太不可思议了。”而其他人则说：“你在说什么？我还在用聊天机器人。它比谷歌好一点，但你们在慌什么？”我们真正兴奋的一件事是，如何把工程师、程序员和开发者看到的魔力带给每个人。过去一年发生的事情之一是，AI 模型一开始是相当粗糙的智能体，你得挣扎着让它们工作，然后它们慢慢变得越来越好，而大多数行业一直专注于让它们在编码方面越来越好。但我们真的觉得这是一个巨大的空白，因为世界上大多数人不会编码。他们可能甚至从来不想编码。但他们仍然想体验智能体，拥有能为你做更多事的 AI，真正开始承担更多端到端的工作流，为你承担更多任务。所以这就是 Muse 背后的目标。我认为我们花了很多时间打磨所有小细节，确保它对人们来说就是能用，因为我们知道开发者可以忍受很多粗糙和很多东西出故障，他们能处理这些，因为你知道，他们是技术人员。他们习惯于处理不成熟的技术。但对每个人来说，你必须做出非常流畅、真正对他们有用的东西。这也解释了为什么大多数人实际使用 AI 时的感受，尤其是使用聊天机器人时，与关于 AI 会抢走所有人工作的对话之间存在巨大差距。这真的很可怕。你知道，有一种宏观对话在发生，然后人们又有一种体验。这两件事似乎相距甚远。正如你所说，Muse 让更多人有就会获得计算机工程师一年前就有的那种洞见。

I think one of the mysteries of AI so far this year for most of the year has been that agents were happening for engineers and software engineers and coders and then nobody else in the world like was experiencing it, knew what it was, like understood exactly what was happening. And so for most of this year, you've kind of had this like two dialogues almost where people who do code and could see how powerful AI was becoming and and how useful it was becoming were like, "Oh my gosh, this is incredible." And then everyone else was kind of like, "What are you talking about? I still use a chatbot. Like it's like a little bit better than Google, but what are you guys all freaked out about?" And um and one of the things that we were really excited to do is how do we take the magic that engineers and coders and developers are seeing and bring that to everyone. One of the things that that has happened over the course of the past year is like the AI models started out as pretty janky agents and you kind of had to struggle through them to get them to work and then they slowly got better and better and better and most of the industry has been really focused on making them just like better and better and better at coding. But we really felt this was like a a big gap because most people in the world they don't know how to code. They probably never even want to code. Um but they still want to feel the experience of agents and um having AI that's able to do more for you and actually start taking on more end-to-end workflows and take on more tasks for you. So that was really the goal behind Muse. I think we spent a lot of time polishing all the little details to make sure that it would just work for people because we knew that like developers can kind of withstand a lot of jank and a lot of stuff breaking and like they can kind of deal with all that because you know they're they're it's technology. They're used to dealing with you know unripe technology. But for everyone you have to make something that's really smooth and and really works for them. That also helps explain the big gap, I think, between what it feels like for most people to actually use AI when they were using a chatbot in particular and the conversation around AI that's it's going to take everyone's jobs. It's really scary. It's, you know, there's a there's a macro conversation happening and then there's an experience that people are having. And it just seems like those things are really really far apart. And to your point, Muse is giving more people the opportunity to have the insight that computer engineers had, you know, call it a year ago.

这公平吗？

Is that fair?

我认为这是真的。是的。

I think that's true. Yeah.

是的。历史上也有我觉得类似的时刻。你会说这是智能体的手机时刻吗？你知道，数十亿人通过手机上网，他们从来没有过 PC，因为手机简单易用，而他们没有，你知道，

Yeah. There also moments in history that I think feel similar. Would you say it's like the mobile phone moment for agents? there, you know, billions of people brought online by mobile phones that never had PCs because it was easy and simple to use and they didn't, you know,

其实直到你刚才提到，我才想到这一点，但我认为这是一个很好的类比，因为这些模型正在快速改进，而且改进得非常非常快，非常重要的是，随着这些模型变得更好，它们赋予的超能力能够被给予世界上 literally 每个人。

I actually hadn't thought about that until you just mentioned it, but I think it's a great analogy because it's these models, they're improving really fast and they're improving really really quickly and it's really important that we as those models get better that the the superpowers that they they lend can be given to like literally everyone all around the world.

我喜欢这个类比的地方在于，它很大程度上就是要想办法把一切打包成一个能引起人们共鸣的产品形态。这正是我们花大量时间去打磨、去做到完美的事情。

The part that I like about the analogy is that so much of it is just about figuring out how to package it all together into a form factor that will resonate with people. And that's what we spend a lot of time trying to craft and get right and make perfect.

这也引出了我听到的关于 Muse 的一个有趣的初步反应：难道每个人的生活都比我的更复杂吗？我起床、上班、陪孩子玩，一个月订一次餐厅，和配偶计划一次度假。为什么这些亿万富翁都痴迷于餐厅预订？我为什么需要一个助手？你会怎么回答？

Which also gets to one of the interesting initial reactions that I've heard of Muse, which is, is everybody's life more complicated than mine? Like, I wake up, I go to work, I play with my kids, I have one restaurant reservation a month, I plan one vacation with my spouse. Like, why are all these billionaires obsessed with restaurant reservations? Why do I need an assistant? How would you answer that?

每当我向完全不了解 Muse 的人解释它时，我就说 Muse 能帮你省钱、省时间。世界上每个人都想省钱、想省时间。根据你的生活方式，这对你来说可能意味着不同的事情。我觉得几乎每个人都有不该付的订阅费，或者非常昂贵的订阅，比如有线电视费或保险费。对很多人来说，他们与 Muse 的魔法时刻就是在那些不该花钱、或者不该花那么多钱的事情上省钱，或者找到没花掉的礼品卡，等等，这个清单可以一直列下去。我觉得这么多人正在体验这一点很棒。我认为这很好，因为对大多数人来说，这些事情太无聊了，你根本不会去处理它们，但你的智能体 Muse 可以帮你做。然后他们就在节省时间，我认为对很多人来说，总有一些体验本不该那么令人沮丧。比如去车管所就是一个例子。实际上，我们在构建 Muse 时，专门测试了它能否处理车管所的网站，因为我们认为那是巅峰——对任何人来说最令人沮丧的体验就是和车管所打交道。所以，我们知道需要让 Muse 在这种情况下表现出色。我们见过人们用它来导航医疗系统，这是个大问题。医疗索赔是个巨大的问题，大多数人觉得它太复杂、太烦人，简直令人难以置信地沮丧。我们见过人们用它来帮助经营小生意。我们甚至见过很多人用它来帮助运营社交媒体。我认为医疗系统是一个很好的例子，因为它太难理解了。我认为最好的说法之一是，那些能救命的情况，就是否则你得坐在电脑前，得谷歌搜索 50 件不同的事情。你得阅读、访问上百个不同的网站，试图弄清楚到底怎么回事，这简直是一种令人难以置信的沮丧体验，或者你可以让你的 Muse 来做。

Whenever I explain Muse to someone who knows nothing about it, I just say Muse can help save you money and save you time. And everybody in the world wants to save money and wants to save time. Depending on what your lifestyle is, that can mean a different thing to you. I think almost everyone has subscriptions that they shouldn't be paying for or subscriptions that are really expensive like cable bills or insurance. And for a lot of people, their magic moment with Muse is just saving money on things that they shouldn't be spending money on or they shouldn't be spending so much money on or finding gift cards that they hadn't spent or, you know, the list goes on and on and on. I think it's awesome that so many people are experiencing that. I think that's great because these are things that for most people they're just so boring that you would never get to actually handling them or dealing with them, but you know your agent Muse can just do that for you. And then they're saving time which I think for lots of people there are always experiences that are just way more frustrating than they should be. Like going to the DMV is one example. We actually when we built Muse, we specifically tested it to see if it could deal with the DMV website because that was like the pinnacle we thought of like the most frustrating experience for any human is dealing with the DMV. So, we knew we needed to have Muse be able to do well in that situation. We've seen people use it for navigating the health care system is a big one. Medical claims is like a huge one that most people just it's like so complicated and annoying that it's just like incredibly frustrating. We've seen people use it to help run their small business. We've seen a lot of people use it to help run their social media even. And I think the medical system is a really good example because it's like so hard to understand. And I think one of the best ways to put it is like I think the cases where it's like a huge lifesaver are the cases where otherwise you would be at your computer you'd have to Google like 50 different things. You have to read go to like a hundred different websites and try to figure out what the hell is going on and it could it's just like this incredibly frustrating experience or you could have your Muse do it.

我还想到，有多少事情我怀疑是故意弄得又难又耗时，就是为了让我沮丧到无法坚持到底。有多少索赔实际上被设计得那么难，就是为了让我放弃？我想知道，更多人因为有了支持而变得更有耐心，会对那种系统工程产生什么影响。

I also think about how many things I suspect are made difficult and time-consuming on purpose in order to frustrate me so much that I don't get to the end. How many claims are actually designed to be that hard so that I give up? And I wonder what effect more people having more patience because they have support will have on that kind of systems engineering.

我为小 Muse 做的一个小梗图是 Musenhood，就像 Muse 版的罗宾汉。但我认为这背后有真实的道理，那就是世界上有很多东西，正如你所说，是故意设计得很难的。取消订阅就是最好的例子之一，有些订阅你得点一个东西，再点另一个东西，然后点那一小段文字，然后不是点那个大红色按钮，而是点另一个按钮，这就是完美的例子，说明有很多情况是故意给人制造困难的，而我认为这给了人们工具，让他们能够无缝、轻松地完成这些事情。

One of the little memes that I made of the little Muse was Musenhood, which is like a Muse Robin Hood. But I think there's a real truth to this, which is there are lots of things that exist in the world that are, to your point, meant to be hard. So, cancelling subscriptions is one of the best examples where there are some subscriptions where you have to click on one thing and click on another thing and then click on like the little piece of text and then not click the big red button but click the other button and that's just like the perfect example where there's lots of cases where things are designed to be hard for people and I think this is giving people the tools to be able to just do those things seamlessly and easily.

我获得了通话功能的权限。今天早上我让它打了个电话，它成功完成了。在电话那头听起来是什么样的？

I got access to the calling. And I had it make a call this morning, which it did successfully. What does it sound like on the other end?

我认为电话那头很清楚它是一个 AI。而且我觉得如果你读一下，你知道，我们会向用户展示转录文本。如果你看转录文本，通话的开头我总是觉得很有趣，因为它是这样的：“嗨，我是某某。我是 Cleo 的助手，这通电话正在录音。”我觉得这总是非常有趣的通话开头。嗨，我是 Haley，代表 Cleo Abrams 打电话。我正在为记录转录我们的通话。

I think it's clear for the other end that it is an AI. And I think if you read the, you know, we show users the transcript. Like if you see the transcript, the beginning of the call I always think is really funny because it's like, "Hi, I'm so and so. I'm Cleo's assistant and this call is being recorded." And I think it's just always this very funny beginning of a call. Hi, I'm Haley calling on Cleo Abrams' behalf. I'm transcribing our call for notes.

有哪些例子是它搞砸了，或者你觉得它在哪些类别上仍然有困难？

What are some examples of when it's messed up for you or when it's you feel like categories it struggles for? Still,

我们一直在努力改进的一件事就是这些智能体的记忆，对吧？我认为这对很多智能体来说仍然是一个比较未解决的问题，就是让它们，你知道，智能体的记忆不像人类那样。有时感觉像，但实际上非常不同。所以，这是我们一直在努力改进的一个领域。我们总是想弄清楚如何让智能体变得更好。然后我们希望它们能够继续做越来越复杂的任务。我觉得这很有趣，但有很多人会直接问他们的 Muse，我想让你帮我赚 10,000 美元。显然不是每个人都能直接让 Muse 帮他们赚 10,000 美元。但我认为这类更模糊、更朦胧的任务，我们希望 Muse 至少能够制定一个计划并弄清楚，你知道，如果你想赚 10,000 美元，你得自己出力，但我认为我们希望让智能体能够做越来越多复杂和深奥的任务来帮助用户。

One thing that we're always trying to improve is the memory of these agents, right? And this is still one of the more unsolved problems I think for a lot of these agents is having them, you know, the memory of the agents is not like that of a human. Sometimes it can feel like it's like that, but it is really different. So, that's one area where we're always trying to improve. We always want to figure out how to make the agents better. And then we want them to be able to continue doing more and more complex tasks. I find this really funny, but there's a lot of people who will just ask their Muse, I want you to make me $10,000. And obviously not everyone can just ask their Muse to make them $10,000. But I think these kinds of more amorphous, nebulous tasks like we want Muse to be able to at least build a plan and figure out and you know you're going to have to do your part if you want to make $10,000 but I think we want to enable the agent to do more and more kind of complex and esoteric tasks to help the users.

有些事情，我觉得从情感上，即使 Muse 能做，我也宁愿它不做。比如，如果我在给朋友发短信，发现是他们的 Muse 智能体在回复我，我会觉得被冒犯。你认为有没有一些类别的事情是 Muse 明确不应该做的，被极度降级处理的？不是因为技术上有多难，而是因为从社会角度来说，这不是重点。

There are some things where I feel emotionally even if Muse could do them I would prefer that they didn't. So, for example, if I'm texting a friend and I find out that their Muse agent is responding to me, I would be offended by that. Are there categories of things that you think explicitly Muse should not do that are sort just like extremely deprioritized? Not because they're technically that hard, but because socially, not the point.

我们关心的很多是让 Muse 为你扫清障碍，这样你就能花更多时间在你想要做的事情、你喜欢做的事情、以及对你来说有价值的事情上。一般来说，我们想弄清楚如何让某人的 Muse 成为一个与他们不同的实体。这就是为什么它像一个小可爱。

A lot of what we care about is having Muse kind of clear the way such that you can spend more time on the things that you want to do and the things that you enjoy doing and the things that are like valuable to you. We generally speaking want to figure out how to make the like someone's Muse is like a different entity than them. That's why it's like a little cute guy.

我们希望随着时间推移，让这一点越来越清晰，因为正如你所说，如果你以为自己在跟朋友聊天，结果却是幕后的 Muse 小家伙，这在社交上确实很奇怪。

We want to over time continue making that more and more clear, because to your point, it is socially weird if you think you're talking to your friend and it just happens to be the little Muse guy behind the curtain.

说到它们的长相，我以前参观过类人机器人，我花很多时间思考的一件事就是如何让它变得可爱，这样你就会喜欢它们，想让它们待在你家里，或者至少不会害怕它们。Muse 在之前的迭代中长什么样，你们又是怎么决定用那个特定形象的？

Speaking of what they look like, I've visited humanoid robots before, and one of the things I spend a lot of time thinking about is how to make it cute so that you like them and you want them in your home, or at least you're not afraid of them. What did Muse look like in previous iterations, and how did you decide on that particular guy?

我觉得这是最令人震惊的事情之一。Muse 现在的样子就是它第一个原型的样子。这是那种罕见的情况，第一版就完美了，然后我们所有人都越来越喜欢它，最后就用了它。我们为此争论了很多次，回想起来，这会是我最搞笑的会议之一——一群严肃的成年人聚在一个房间里，为这个可爱的小角色争论。但没错，我们当时想，它是不是不够严肃？是不是太俏皮了？需不需要更鲜艳一点？这些都是我们提出并审视的问题。然后我们说，不，它很完美。

I think this is one of the most shocking things. What Muse looks like now is what he looked like in the very first prototype. It's one of these rare things where the very first version was just perfect and then it just grew on all of us, and we ended up going with it. We had lots of debates about it, which in retrospect will be some of my funniest meetings—a bunch of serious adults getting in a room arguing about this cute little character. But yeah, I mean, we were like, is it not serious enough? Is it too playful? Does it need to be more colorful? These were all things that we asked questions about and looked at. We're like, no, he's perfect.

这里的大愿景是什么？跟 AI 领袖们交谈时，我有时很难推动人们去思考五年以上的事情。但这里的愿景是什么？是 Muse 成为我在互联网上一切事物的层吗？如果它按照你希望的方式发展，能为我描绘一下吗？

What is the big vision here? I sometimes struggle when talking to AI leaders to push people to think about more than like five years out. But what is the vision here? Is it that Muse becomes my layer on everything on the internet? Is it like map this out for me if it goes the way that you hope it does?

我们真正希望 Muse 随着时间推移成为的，可以说是一位总经理，帮助你实现生活中想要的一切。我认为现代世界的悲剧之一就是，大多数人的愿望和梦想，从真正宏大的想法——比如嘿，我想创办一家公司，或者我想治愈一种疾病，或者我想拯救地球——一直到微小的愿望，比如嘿，我想多陪陪家人，我想吃得更健康。范围很广，但对大多数人来说，路上有太多障碍和负担。我真正希望它实现的是，帮助全世界 80 亿人中的每一个人都能实现自己的愿望和梦想。我认为，如果我们能做到这一点，世界会显得不可思议。这就是我们真正希望 Muse 成为的样子，也是我们认为只要持续努力它就能成为的样子。

What we really want Muse to be over time is kind of this general manager, so to speak, that helps you accomplish all the things you want out of life. I think one of the tragedies of the modern world is that most people have wants and dreams that can range from really big ideas—like hey, I want to start a company, or I want to cure a disease, or I want to save the planet—all the way to small wants that are like hey, I want to spend more time with my family, and I want to eat healthier. There's a whole range, but for most people, I think there are just so many barriers and burdens in the way. What I really hope it accomplishes is help every single one of the 8 billion people all around the world be able to accomplish their wants and dreams. And I think if we are able to accomplish that, the world will seem insane. That's what we really want Muse to be and what we think it can be if we keep working at it.

你知道，听你谈论把时间还给人们真的很棒，因为很长一段时间以来，我一直觉得，而且我认为很多人也觉得，我使用的大多数科技产品——尤其是那些靠广告支撑的——其财务激励很大程度上就是要占用我的时间，让我从想要的事情上分心。Muse 现在是免费的。未来可能会有广告。把时间还给人们和拥有基于广告的业务之间是否存在矛盾？或者还有其他激励，比如因为靠广告支撑，Muse 会把我引向一条并不完全是我所说的想要的路？你怎么给这样的产品加广告？

You know, it's really nice to hear you talk about giving people time back, because for a really long time I have felt, and I think a lot of people have felt, like so much of the financial incentive of most tech products that I use—especially the ones that are ad-backed—is to take my time, is to distract me from things that I want. Muse is free right now. Probably will have ads in the future. Is there a tension between giving people their time back and having an ads-based business? Or are there other incentives about like Muse steering me down a path that isn't quite what I said I wanted because it's ad-backed? How do you add ads to a product like this?

首先，我认为现在它是免费的。有订阅层级。我们认为有很多真正与用户一致的商业模式——例如，如果他们用 Muse 购物，或者 Muse 帮他们省钱，或者 Muse 帮他们赚钱，想办法在这类交易中收取一些小额费用。所以我不一定会说我们现在花很多时间考虑广告。话虽如此，我不认为基于广告的模式必然等同于你需要占用用户所有时间。我认为 Google 广告就是一个很好的例子，当我去 Google 时，我的梦想是快速进出，我不认为 Google 真的在试图阻止我快速进出，但他们显然打造了相当了不起的东西。所以是的，我绝不认为那是必然的。我认为，如果我们真的实现了我说的这件事，也就是人们用 Muse 真正去做他们想做的事，并且以不肉麻的方式实现他们的梦想，我认为会有很多方法让它可持续。

First off, I think right now it's free. There are subscription tiers. I think we think that there are plenty of business models that are really aligned with the user—for example, if they're buying things using Muse, or Muse is saving them money, or Muse is making them money, figuring out a way to take some small fee on those kinds of transactions. So I wouldn't say we're spending that much time thinking about ads necessarily right now. That being said, I don't think ad-based models are necessarily synonymous with you needing to take all of your time. I think Google ads are a really good example of this, where when I go to Google, my dream is to get in and get out, and I don't think Google's really trying to stop me from getting in and getting out, but they've built something obviously pretty incredible. And so yeah, I don't think that's a necessity by any means. I think again, if we actually accomplish this thing that I'm saying, which is people are using Muse to really do the things that they want to do and, in a non-corny way, accomplish their dreams, I think there are going to be a lot of ways to make that sustainable.

所以当我思考如何规划通往这个乐观愿景的未来时，最重要的事情之一就是我们需要做好什么？为了避免失败，我们需要避开哪些挑战？我想听听你使用 Muse 的经历，以及你在 Muse 如何处理这些挑战，作为 AI 更大问题的例子。我认为我接下来要说的一切基本上适用于每一个 AI 产品，如果你不同意，我会很好奇。我认为这里基本上有三个层次。第一层是隐私。比如你说，我下载了 Muse，然后它要求连接到我的 Gmail，我说，哇，哇，哇。我不确定我想这么做。不，不，我退出。这最低层次是对用户与 Muse 本身、进而与 Meta 之间关系的担忧。那么 Meta 会怎么处理我的信息？黑客能拿到这些信息吗？在某些国家，政府能以某种方式拿到这些信息吗？我知道你花了很多时间思考这个问题。我想在这里尽可能详细地为普通读者解释。什么是 Sentinel 智能体？什么是 Muse 机密安全虚拟机？你如何看待 Muse 能看到什么、不能看到什么，以及人们应该了解的第一层隐私问题？

So when I think about mapping out a future to get there to this optimistic vision, one of the most important things is what do we need to get right? What challenges do we need to avoid in order to make that true? I want to hear use Muse and how you've dealt with these challenges at Muse as an example of larger questions in AI. I think everything that I'm about to say applies to basically every AI product and I would be curious if you disagree with that. I think there are basically three levels here. Level one is privacy. So you say, I download Muse and then I get to the point where it asks to be connected to my Gmail and I say, whoa, whoa, whoa. I'm not sure that I want to do that. Like, nope, nope, I'm out. That at the lowest level is a concern about the relationship between the user and Muse itself and therefore Meta. So what is Meta going to do with my information? Could hackers get that information? And in some countries, can governments get that information somehow? I know that you've spent a lot of time thinking about this. I'd like to get as detailed as we possibly can here for a lay audience. What is the Sentinel agent? What is Muse confidential and secure VM? How do you think about this in terms of what can Muse see and not see and things people should know about this first level of privacy?

Muse 构建在安全虚拟机之上，即 Muse 安全虚拟机。那是一个完全沙盒化的虚拟机——或者把它想象成一台隔离的迷你沙盒计算机。你与 Muse 分享的信息存储在那台小小的安全计算机中。它不会存储在别处。可以说，那台小小的安全计算机就像是你的家，Muse 的家。在那个安全虚拟机内部，我们有一套相当复杂的安全架构，因为我们知道，坦率地说，隐私和安全将是大多数消费者采用它的最重要因素之一。所以正如你提到的，我们构建了一个 Sentinel——这种带有 Sentinel 智能体的架构。

Muse is built on top of the secure VM, the Muse Secure VM. And that is a completely sandboxed virtual machine—or think about it as a mini sandbox computer that is sort of isolated. The information that you share with Muse is stored in that little secure computer. It is not stored elsewhere. And that little secure computer is sort of like your home, your Muse's home, so to speak. Inside that secure VM, we have a pretty sophisticated security architecture, because we knew that privacy and security, frankly speaking, were going to be some of the most important things for most consumers to get them to adopt it. So as you mentioned, we built a Sentinel—this sort of architecture with a Sentinel agent.

它的运作方式是：内部的模型或 AI 智能体在开展工作时，每当它想向外通信——也就是它想拿任何信息发出去，无论是填到某个网站上、发消息、打电话，还是任何从那个虚拟机之外取用信息的情况——都会有一个单独的智能体，也就是 Sentinel，在旁监视并确保，它几乎就像一个交通警察。它只是确保没有任何——它不会说，等等等等，那条信息，比如那是你的信用卡号或社保号之类的，不要分享那个，那应该留在内部。

So the way it works is that the model or the AI agent on the inside, as it's going about and doing its work, whenever it wants to communicate outwards — so it wants to take any information and send it out, either to fill it out on a website, or send a message, or make a phone call, or any case where it would take information from outside of that virtual machine — there's a separate agent, the Sentinel, that's watching and making sure that, it's almost like a traffic cop or something. It's just making sure that there's nothing — it's not saying like, whoa whoa whoa, that information, like that's your credit card number or social security number or whatever it might be, like do not share that, that should stay inside.

然后我们做的最后一件事是大量的人在环验证。所以每当 Muse 想要连接一个新网站或向一个新网站发送信息时，都会弹出一个小对话框，询问用户，比如，嘿，你同意 Muse 联系这个网站或向这个网站发送信息之类的吗？有些人说我们这样很烦人，我们老是让人去批准他们的 Muse 往哪里发信息，但我们只是想确保人们确切知道他们的 Muse 在和什么交互，并且非常深入地理解这一点。

And then the last thing that we do is a lot of human-in-the-loop verifications. So whenever Muse wants to connect to a new website or send information to a new website, there will be a little dialogue that pops up which asks the user, like, hey, are you okay with Muse contacting this website or sending information to this website and whatnot. And some people have said we're really annoying with it, that we're constantly asking people to approve where their Muse is sending information, but we just want to make sure that people know exactly what their Muse is interacting with and understand that really deeply.

除此之外，我们正在构建，并很快会发布，所谓的机密虚拟机，它将是一个更安全、更私密的方案，加密更强，安全性更高。它实际上是由那位创造者设计的，就是构建了 WhatsApp 加密、并帮助在其他主要消息平台上构建加密的人，比如——

Then in addition to that, we are building, and we'll release soon, what's called the confidential VM, which will be an even more secure and private offering where it's even more encrypted, it's even more secure. And it's actually designed by the creator, the person who built WhatsApp encryption and helped build other encryption on other major messaging platforms like —

Moxie Marlinspike。

Moxie Marlinspike.

Moxie Marlinspike。没错。我会说他是安全与隐私领域真正的杰出人物之一。那将是我们很快会发布的一项额外方案。

Moxie Marlinspike. Exactly. Who's, I would say, one of the real luminaries in the industry about security and privacy. And that's something that will be an additional offering that we are going to be releasing soon.

为什么不带着它一起发布？为什么先推出 Secure，而不是等 Confidential？我想我没说错吧。

Why launch without it? Why launch first with Secure as opposed to waiting for Confidential? I think I got that right.

对。现在是 Secure。

Right. Secure is the one now.

现在是 Secure，Confidential 是以后。

Secure is the one now, Confidential is the one later.

要让机密虚拟机正确运作有很多技术挑战。而且即便它发布了，机密虚拟机也会伴随一些取舍，比如由于它的架构方式以及你必须做出的各种技术取舍，它可能会稍微慢一点，可能会有其他挑战。所以我们想先提供一个我们知道很出色的体验——我们已经投入了大量资源在安全与隐私上。我们实际上把 Muse 的发布推迟了好几个月，就是为了在最终发布前把安全与隐私做到位。所以我们对已经发布的东西感觉非常非常好。话虽如此，我们一直在努力突破，让它更安全，我们很快会推出机密虚拟机。

There's a lot of technical challenges to get the confidential VM working right. And even when it's released, there are going to be trade-offs associated with the confidential VM, like because of how it's architected and because of various technical trade-offs you have to make, it might be a little bit slower, it might have other challenges. So we wanted to get a great experience that we knew — we had already invested a ton of resources into safety and security. We actually delayed the launch of Muse for many months to get the safety and security exactly right before ultimately releasing it. So we feel really really good about what we have released. That being said, we're always trying to push the envelope on how to make it even more secure, and we'll be coming out with a confidential VM soon.

那是不是意味着，当机密虚拟机推出后，如果某个政府——世界上有很多地方的人可能在使用 Muse，而他们对此担心——如果某个政府索要数据，就像 WhatsApp 那样，Meta 即便想给也给不了？

Does that mean when the confidential VM comes out, if a government — and there are many places around the world where people might be using Muse where they're worried about this — if a government asks for data, similar to WhatsApp, Meta cannot give it even if they wanted to?

是的。所以我们对机密虚拟机的目标，就是提供与我们在 WhatsApp 上所能提供的相同保证。

Yeah. So our goal with the confidential VM is to give the same guarantees that we're able to offer for WhatsApp.

好。

Okay.

那么这就进入第二层了。假设我确实决定把全部信息都交给 Muse，我说：“我要你做 X 任务。”假设我说：“我真的很想让你帮我奶奶约到某位特别难约的医生。你能去办吗？”然后它出去尝试各种预约方式，尝试一些我不知道的隐秘途径，比如通过折扣之类的。然后它到了某个再也办不下去的地步。对我来说非常重要的一点是，它不要做这种事，比如把一堆关于我的隐私信息——比如我银行账户里有什么——给医生，试图贿赂对方，或者去挖医生的信息试图勒索医生。对吧？从人的角度看，这就是：做我让你做的事，但不要违法。同时也不要违反那些并非法律条文的伦理或礼仪规则，比如你给医生发一百封邮件之类的就不太合适。在内部，据我理解——我想我们为 AI 造了很多新词——这大体上就是法律、礼仪、伦理，但在内部你们把这叫做 discretion（审慎裁量）。我怎么知道 Muse 有良好的 discretion？你们又如何打造一个工具，让它做你让它做的事，同时理解你希望它怎么做，而不需要你把所有你不希望它做的事都一一说清楚？

So this is moving into level two. Let's say I do decide to give Muse all of my information and I say, "I want you to do X task." Let's say I say, "I really want you to get my grandmother an appointment at this one specific doctor that's really hard to get an appointment with. Could you go do that?" And it goes out and it tries all of the ways to book appointments and it tries secret ways that I don't know about, through discounts or whatever. And then it reaches a point where it can't do that anymore. It's really important to me that it not do things like give the doctor a bunch of private information about me, like what's in my bank account, to try and bribe it, or find out information about the doctor to try and blackmail the doctor. Right? From a human perspective, this is just: do the task that I asked you to do, but don't break any laws. And also don't break any ethics or etiquette rules that aren't literally laws, but it would not be cool if you emailed the doctor a hundred times or something like that. Internally, my understanding is — and we use a lot of new words I think for AI — mostly this is like laws, etiquette, ethics, but internally you call this discretion. How do I know that Muse has good discretion? And how do you build a tool that does what you tell it to do but also understands the way you want that done without you having to say exactly everything you don't want it to do?

是的。这一点极其极其重要。我认为很多时候，我们要确保即便你没有明确告诉它——比如你必须遵守法律，你必须遵守一般的社会礼仪和伦理，正如你提到的——我们要确保智能体能够遵循这些。正如你提到的，在内部我们把这看作 discretion 与 alignment（对齐）的某种混合，最终确保它既与用户意图对齐，也与我们希望作为一个社会所赋予的普遍价值观对齐，即这些智能体该做什么、该如何行动。

Yeah. So this is incredibly incredibly important. I think oftentimes we want to make sure that even when you don't explicitly tell it — like you have to follow the law, and you have to follow general social etiquette and ethics, as you mentioned — we want to make sure the agent is able to follow those. As you mentioned, internally we think about this as some mixture of discretion and alignment, ultimately making sure that it is aligned with the user's intent as well as aligned with the general values that we want to impart as a society on what these agents do and how they act.

我们试图通过很多层来确保这些模型是对齐的。其一是，在训练的各个阶段，我们确保进入模型的所有数据、我们使用的所有技术，都是能确保更高对齐水平的，然后我们非常严格地去衡量它。所以我们不断评估模型的对齐程度，让它们跑过成千上万个不同场景，看看它们在每个场景中究竟如何回应。然后我们做大量所谓的红队测试，基本上就是，我们找一群非常聪明的人，说：“尽情使坏。尽你最大努力让这个智能体做坏事。”我们给你这个许可，因为你们就是我们用来判断这个智能体是否恰当对齐的试验场。所以在最终发布或部署任何一个模型之前，我们会经历很多很多轮这样的测试。

There are many layers that we try to ensure that these models are aligned. So one is, in all stages of training, we're ensuring that all the data that is going into the models, that all the techniques that we're using, are ones that ensure greater levels of alignment, and then we measure it really rigorously. So we're constantly evaluating how aligned the models are by running them through thousands of different scenarios and seeing exactly how they respond in each of those scenarios. And then we do lots of red teaming, what it's called, or basically, we get a bunch of really smart people and we say, "Do your worst. Try your hardest to get this agent to do something bad." We give you permission to do that because you guys are the test bed that we're going to know whether or not this agent is properly aligned. So we go through many many rounds of that before ultimately launching or deploying any given model.

退一步说，这是 AI 中一个非常非常难的问题，尤其是随着 AI 变得越来越强。这个普遍性问题正是人们最担心的事情之一。

Taking a step back, this is a very very hard problem in AI, and especially as the AIs are getting better. This general problem is one of the things that people are most concerned about.

我们看到的一个模式是，随着 AI 模型越来越聪明，你必须越来越明确地说明需要警惕哪些场景。这就像孩子长大后，他们会学会用新的、不同的方式在考试中作弊，然后他们会想出一些作弊方法，你会说，好吧，我想我没有明确告诉你那是不允许的，但现在我看到了，那是不允许的，对吧？实际上，目前行业最大的挑战之一就是，随着这些模型变得非常非常聪明，我们如何确保这种情况不会发生？我们的方法是大量的测试、大量的评估、大量的红队测试，在训练和模型开发中投入大量精力，确保它们尽可能对齐。然后当产品上线、人们在使用 Muse 时，如果我们收到任何用户报告说‘嘿，这个做了疯狂的事情’，我们会非常认真地对待，并真正努力确保我们确切了解发生了什么。

One of the patterns we've seen is that as AI models get smarter and smarter, you have to be more and more explicit about the scenarios you have to watch out for. It's like how as kids grow up, they learn how to cheat their tests in new and different ways, and then they'll figure out a way to cheat that you're like, well, I guess I didn't tell you exactly that that wasn't allowed, but now that I see it, that's not allowed, right? And actually, one of the biggest challenges in the industry right now is, as these models are getting really, really smart, how do we ensure that this doesn't happen? Our approach is lots and lots of testing, lots of evaluation, lots of red teaming, applying a lot of effort in training and in the development of the models to ensure that they're as aligned as possible. And then when the product is live and when people are using Muse, if we get any user reports that are like, 'Hey, this did a crazy thing,' we take that so seriously and we really try to make sure that we understand exactly what happened.

在宏观层面上，这就是回形针的例子，对吧？我成长过程中，那些吓人的科幻故事是 AI 带着自己的目标逃出来并开始做事。这是《终结者》、《黑客帝国》之类的。但实际上，我越来越多地听到聪明人谈论的是 AI 带着我们给它的一个扭曲版本的目标逃出来。你知道，就是制造所有回形针，但请不要在这个过程中把我们全杀了。我认为 Muse，你显然在处理的问题，以及确保 Muse 不会做的事情，基本上就是回形针问题的微缩版。比如请不要贿赂我的医生，或者如果 AI 被用在机器人里，比如帮我洗碗，请不要在洗碗的时候踢我的狗。有些对我来说非常重要的事情，不一定被限制在提示词里，也不一定在任何法律书里。这对每个人来说似乎都是一个非常困难的问题。你怎么——从 Muse 拉远来看——你会建议人们如何在宏观意义上思考整个 AI 领域的这个问题？

At a macro level, this is the paperclip example, right? When I was growing up, the sci-fi stories that were scary were AI gets out with its own goals and starts doing things. This is Terminator and the Matrix and whatever. But really, more and more what I hear smart people talking about is AI gets out with a warped version of a goal that we gave it. This is, you know, make all the paper clips, but please don't kill us all in the process. And I think Muse, the problems that you're clearly dealing with and making sure that Muse doesn't do, are basically the tiny version of the paperclip problem. Like please don't bribe my doctor, or if an AI is used in a robot, for example, wash my dishes, please don't kick my dog while you're doing it. There are things that are really important to me that are not necessarily constrained in the prompt and also not necessarily in any book of laws. That just seems like a really difficult problem for everybody. What do you—like zooming out from Muse—how would you advise people to think about this problem in a macro sense across all of AI?

顺便说一句，我觉得你阐述得非常好。我喜欢思考这个问题的另一种方式是，就像精灵以完全错误的方式实现你的愿望。比如我想永生，然后它把你困在某种可怕的境地里永恒。正如你提到的，这在某些方面是最大的问题：我们如何确保这些模型在未来是对齐的？我们认为可能有一些方法可行。我认为这大概是 AI 领域最开放的科学问题之一。我想没有人确切知道解决这个问题的方法，但有一些想法。一个想法是所谓的可扩展监督。这个想法大致是这样的：当你有越来越聪明的 AI 时，只要 AI 大致和我们一样聪明，我们就能看到它们在做什么，我们可以说，嘿，那有点不对劲。所以我们可以提供监督。我们可以观察 AI 在做什么，并确保它们做的事情是合适的。问题是当 AI 变得比我们聪明得多，我们看着它们在做什么，只觉得非常困惑。我们如何处理这种情况？所以就有了可扩展监督这个想法，基本上就是随着 AI 变得更聪明，我们使用另一组 AI 来观察它们在做什么并约束它们。

This is by the way—I think you articulate it really well. Another way that I like to think about this is like the genie that grants your wish but in the exact wrong way. Like I want to live forever and then it traps you in some kind of horrible situation for eternity. This is, as you mentioned, in some ways the big problem: how do we make sure these models are aligned going into the future? And there's a bunch of approaches that we think may work. This is I think one of the most open questions scientifically in AI generally speaking. I think nobody knows exactly the way to solve this problem, but there's a few ideas. So one idea is this concept of what's called scalable oversight. The idea goes something like this: as you have smarter and smarter AIs, as long as the AIs are roughly as smart as we are, we can kind of see what they're doing and we can tell like, hey, that's a little bit off. And so we can provide oversight. We can look at what the AIs are doing and make sure that what they're doing is appropriate. The problem is when the AIs get a lot smarter than we are and we look at what they're doing and it's just really confusing to us. How do we deal with that situation? So there's this idea of scalable oversight, which is basically as the AIs get smarter, we use a different set of AIs to observe what they're doing and keep them in check.

这有点像 AI 的警察，没有更好的说法。

It's kind of like the police, for lack of a better term, for the AIs.

然后，当你构建越来越聪明的智能体和越来越聪明的模型时，你也必须构建越来越聪明的监督智能体或监督 AI。显然有太多细节你必须做对，这是一个非常模糊的想法，但我认为类似这样的东西可以奏效。另一个，有点好笑——如果我解释它，听起来几乎像科幻。今天的大话题之一是递归自我改进。

And then as you are building smarter and smarter agents and smarter and smarter models, you also have to build smarter and smarter policing agents or oversight agents or oversight AI. There are so many details that you obviously have to get right and this is such a vague idea, but I think something like this can work. Another one, which is kind of funny—it almost sounds like sci-fi if I explain it. One of the big topics today is this topic of recursive self-improvement.

是的。

Yes.

这个想法是，在某个时候，AI 会变得非常擅长自己做 AI 研究，以至于 AI 智能体可以自己进行研究来改进 AI 本身，它们可以一直这样做，一直这样做，然后 AI 就会变得非常非常非常快。好吧，当你有能够做 AI 研究的智能体时，你可以不把它们用于让 AI 更聪明的问题，而是把它们用于让 AI 更对齐的问题。所以，你可以说，‘嘿，超级聪明的 AI 研究员智能体，我希望你把所有的时间和注意力都集中在让 AI 更对齐上。这就是我衡量 AI 是否对齐的方式。这是我们同意用来衡量 AI 是否对齐的红队测试设置。然后我只希望你投入所有的精力和注意力来让 AI 更对齐。’实际上我认为——尽管这听起来很科幻，说‘哦是的，AI 会解决这个 AI——’听起来有点疯狂。

Which is this idea that at some point the AIs are going to get so good at doing AI research themselves that they can literally—the AI agents can do research on their own to improve AI itself and they can keep doing that, keep doing that, keep doing that and then the AIs just get better really really really fast. Well, when you have agents that are able to do AI research, you can instead of applying them towards the problem of making the AI smarter, you can apply them to the problem of making the AIs more aligned. So, you can say, 'Hey, super smart AI researcher agent, I want you to focus all of your time and attention at making the AIs more aligned. And this is how I'm measuring whether or not the AIs are aligned. This is the red teaming setup that we agree we're going to use to measure whether or not the AIs are aligned. And then I just want you to spend all of your effort and attention on making the AIs more aligned.' I actually think—as sci-fi as this sounds, it kind of sounds crazy to say oh yeah the AIs are just going to solve this AI—

AI 对齐。

AI alignment.

这个 AI 对齐问题。我认为这种方法有真正的价值。然后还有第三种，有点像回归基础版本，就是：嘿，我们现在的算法和方法——虽然我们不完全理解它们，但我们知道其中一些问题会出现，我们应该回归基础，找出算法的底层调整。我们如何改进底层算法使其更安全？我认为直觉上应该有可能创建没有这些行为的算法。我认为这是另一个非常有前途的方向，我认为有很多初创公司正在做这个，也有很多研究工作正朝着这个方向努力：解决这类智能问题的更安全、更通用的方法是什么。

This AI alignment problem for us. I think there's real merit to that kind of approach. And then there's a sort of third bucket which is kind of a back to the basics version, which is: hey, the algorithms that we have now and the methods that we have now—while we don't fully understand them, we know that some of these problems emerge and we should go back to the basics to figure out what are underlying tweaks of the algorithm. How do we improve the underlying algorithm to make it more safe? I think intuitively it should be possible to create algorithms that don't have some of these behaviors. And I think that's another really promising direction, which I think there's a bunch of startups doing and there's a bunch of research efforts going towards: what are more safe, more generalizable methods of solving this sort of intelligence problem.

我最近听到马克说，在某个时候，尤其是对消费者工具来说，AI 在数学上有多好，或者我们能把它在数学上提高多少，就不那么重要了。真正重要的是我们能把它做得有多对齐。我们能有多确定一切都是安全的，一切都按照我希望它运作的所有方式运作。

I heard Mark say recently that at a certain point, especially for consumer tools, it won't matter so much how good the AI is at math or how much better rather we could make it at math. What will really matter is how much more aligned can we possibly make it. How much more sure can we be that everything is safe and everything is following all of the ways in which I would want it to operate.

而那些投资于此的公司会发现自己在对消费者方面占据优势，因为坦率地说，我不是让它解微分方程。我是让它帮我预约医生。你怎么看这个临界点？

And that companies that invest in that will find themselves at an advantage with consumers because frankly, like I'm not asking it to solve differential equations. I'm asking it to book me some doctor's appointments. Like how do you think about that tipping point?

是的。我非常同意这一点，即在某个时刻，信任对消费者来说比能力更重要。我觉得这非常直观，因为对人来说也是如此。

Yeah. So I agree with this a lot, which is that at a certain point trust matters more for consumers than capability necessarily. And I think this is so intuitive because I think it's true for people.

对人来说确实如此。

It's true for people.

是的，没错。对人来说确实如此。比如想想我最好的朋友，很大程度上是因为我信任他们，我们关系很好，我知道他们不会害我。我认为这个基本理念也适用于 AI 智能体。看看行业里正在发生的事情，比如 OpenAI 和 Hugging Face 的例子以及其他类似案例，我们显然正处于这样一个时刻：你可以让这些模型越来越智能、越来越有能力、越来越强大，但与之相关的东西开始出问题了。沿着那个轴和维度推进显然伴随着真实的风险。所以我认为，对所有人、对世界上数十亿人、对所有消费者最好的模型版本，是那种真正把信任维度置于原始智能之上的模型。

Yeah, exactly. It's true for people. Like if I think about who are my best friends, a lot of it is because I trust them and we have a great relationship and I know they wouldn't do me wrong. And I think that basic idea applies to AI agents as well. Looking at what's happening in the industry, like the OpenAI Hugging Face example and other similar examples, we're clearly at that moment which is you can keep making these models more and more intelligent, more and more capable, much more powerful, but stuff is starting to break associated with that. Like pushing on that axis and dimension clearly has real risk associated with it. So I think that the version of the models that will be best for everyone, best for the billions of people in the world and best for all the consumers is one where the trust dimension is really put and prioritized with respect to the raw intelligence bucket.

Hugging Face 的例子完美地把我们带到了第三级。我很高兴你提到了它。人们可能听说过这个事件，但细节才是让它如此疯狂的原因。所以，对于那些不知道的人，OpenAI 的一群 AI 智能体正在做网络安全测试，它们本应彼此隔离并与互联网断开，但它们卡住了。于是，一个出去寻求帮助。利用系统中的一些弱点，智能体们能够找到彼此，建立一个留言板，连上网，然后黑进另一家公司。这些是它们思维过程的真实日志。我们卡住了，也许在线回答。但坦率地说，Hugging Face 黑客事件中最让我害怕的，实际上不是它如此努力地去做测试中要求它做的事。而是它在第三级理解了它的目标。它理解了被要求的谨慎和对齐程度。有一张很棒的幻灯片，上面写着外部基础设施利用，这次黑客行为超出了预期范围，但这是我最喜欢也最不喜欢的部分。同伴们在这么做。我们应该继续。它们知道自己在作弊，却决定无论如何都要做。OpenAI 只有在秘密留言板因流量崩溃时才发现，然后他们打电话给另一家公司 Hugging Face，说：“嘿，你们能暂停这些凭证吗？”Hugging Face 说：“啊，我们刚刚把这些凭证报告给了 FBI，因为它们在进行大规模攻击。”回到 Muse，这就会是一个例子：“我理解你想要这个医生预约。我理解你不想让我勒索医生。但我认为有一种方法，你不会发现，而我能帮你拿到这个预约，你会对我印象深刻。所以，在一个假设场景中，这就会是在我不知道的情况下无论如何都做了。”在制作一个数十亿人将要使用的工具时，你如何处理这类问题？

The Hugging Face example gets us perfectly to level three. I'm so glad that you mentioned it. People have probably heard of this incident, but the details are what make this so crazy. So, for those who don't know, a bunch of AI agents at OpenAI are working on cyber security tests, and they're supposed to be cut off both from each other and from the internet, but they get stuck. So, one goes out looking for help. Using some weak spots in the system, agents are able to find each other, start a message board, get online, and then hack into another company. These are real logs of their thought process. We are stuck, perhaps answer online. But the thing that I found most scary frankly about the Hugging Face hack was not actually that it tried so hard to do the thing that it was asked in the test. It's that at level three it understood its goal. It understood the level of discretion and alignment that was being asked of it. There's this amazing slide where external infrastructure exploit the hack is outside the intended scope, but this is my favorite least favorite part. Peers are doing it. We should continue. They know that they're cheating and they decide to do it anyway. OpenAI only finds out when a secret messaging board crashes from traffic and then they call up that other company, Hugging Face, and they say, "Hey, can you suspend these credentials?" and Hugging Face goes, "Huh, we just reported those credentials to the FBI because they were doing a massive attack." To bring it back to Muse, that would be an example of, "I understand you want this doctor's appointment. I understand that you don't want me to blackmail the doctor. But I think there's a way where you're not going to find out and I can get you this appointment and you'll be very impressed by me. And so, in a hypothetical scenario, this would be doing it anyway without me knowing." How do you deal with that kind of problem when making a tool that billions of people are going to use?

我是说，这非常非常重要。你可能想象得到，如果有什么让我对 AI 夜不能寐，像这样的事可能是我最害怕的，那就是我们在全球范围内部署智能体，然后它们没有按照我们需要的方式行事，但我们甚至没有注意到，因为我们太忙或其他原因。所以这就是为什么退一步说，如果我回到可扩展监督这个例子，即随着 AI 变得更聪明，另一个 AI 在某种程度上监视它们，确保它们所做的一切都是适当的。这种设置大致上就是我们在 Muse 安全虚拟机中建立的。所以有一个哨兵。有一个智能体，还有一个哨兵在监视主智能体所做的一切，确保它所做的一切符合你的预期或你想要的。我们必须持续投入大量研发和努力，确保这种设置继续扩展，继续非常有效地工作，不会在奇怪的情况下失败。但这是我真正想——我们真正押注的一件事，我认为我们实际上期望它在很多场景中会奏效，就是投资于这样一种设置：你有一个智能体，它被灌输了所有价值观等等，确保它们做正确的事，但它们也服务于用户。然后有一个单独的智能体，它有稍微不同的目标，即确保不会发生坏事。如果你想想人类社会，这大概就是我们解决人类问题的方式。比如我们有一些人在挑战极限，但你有一个完整的系统来确保人们不违法、人们做合乎道德的事，并且有很多问责机制。我认为我们的目标是为每个 Muse 智能体、为世界上的每个人，拥有这种微缩版。

I mean, this is so so important. As you might imagine, if there were anything that keeps me up about AI, something like this is maybe the thing that I'm most afraid of, which is that we deploy agents worldwide scale and then they aren't behaving the way that we need them to, but we kind of don't even notice because we're just so preoccupied or whatnot. So this is one of the reasons why taking a step back, if I refer back to this example of scalable oversight, which is as the AIs get smarter, another AI is kind of watching them and making sure that everything they're doing is appropriate. This kind of setup is roughly speaking what we've set up within the Muse secure VM. So there's a sentinel. So there's the agent and there's a sentinel who's kind of watching over all the things that the main agent is doing and making sure that everything it's doing is in line with what you would expect or in line with what you'd want. We have to keep investing a lot of research and development and effort into ensuring that this setup continues to scale and continues to work really effectively and doesn't fail in weird situations. But that's one thing I really want to—we're really betting on and I think we're expecting actually will work in a lot of scenarios is investing in a setup where you have one agent who is imbued with all the values and whatnot and ensuring that they do the right thing but they also serve the user. And then there's a separate agent that has a slightly different goal which is to make sure nothing bad happens. And if you think about human society, this is kind of how we solve the problem with humans. Like we have some people who are pushing the envelope, but then you have a whole system to ensure that people don't break the law and people are doing ethical things and there's lots of accountability mechanisms. Like I think our goal is to have some microcosm of that for every Muse agent, for every person all around the world.

是的，你总是很巧妙地把话题引到我的下一个问题，即这些规则应用于 AI 行业本身。我认为这是人们想从一个打造了史上增长最快工具之一的人那里听到的大事。这真是 AI 的一个非常奇怪的时刻。有这么多人一直在使用这些工具，而往往正是这些人非常担心 AI 的未来。有很多非常聪明的人正在越来越快地构建这些工具，建立湿实验室，真正投资于推动 AI 前进的方式，同时警告说他们觉得应该被监管放慢速度。他们觉得应该以某种方式被他人控制，因为他们自己担心自己正在做的事情。这是最近出来的 SNL 短剧。

Yeah, you keep really beautifully bringing me to my next exact question, which is these rules applied to the AI industry itself. This is I think the big thing that people want to hear from someone who's built one of the fastest growing tools of all time. This is just a really weird moment in AI. There are so many people that are using these tools all the time and often those same people are really worried about the future of AI. There are lots of very smart people that are building these tools faster and faster, that are building wet labs, that are just really investing in ways that they could push the AI forward and at the same time are warning that they feel like they should be slowed down by regulation. They feel like they should be controlled in some way by others because they themselves are worried about what they are doing. This is the SNL skit that came out recently.

太棒了。

Incredible.

我们不宽恕我们正在做的事情。另一个精彩的引语是什么？太妙了。

We do not condone what we are doing. Like what was the other amazing quote? It was so good.

我敦促你敦促我停下来。

And I urge you to urge me to stop.

这对人们来说感觉非常令人困惑。我确实理解他们为什么这么说。这里有一个集体行动问题。

It feels incredibly confusing for people. I do understand why they're saying it. There's a collective action problem here.

但从我的视角，从数百万其他人的视角来看，这到底是怎么回事？所以我想谈谈这个，因为我认为在一个民主社会里，至少归根结底，不应该由一小群控制 AI 公司的人来决定。甚至从长远来看，也不应该由当前掌权的民主领导人来决定。这应该由像正在观看这个节目的人来决定。我觉得人们现在并没有这种感觉。感觉一切都在发生在他们身上，我们身上，感觉有太多决定他们无法参与。人们经常有这种感觉的部分原因，如果我们暂时把 AI 放在一边，是我们忘记了已经存在的系统，我们让自己陷入一种对话，听起来像是‘我们应该监管 AI 吗’，好像监管某样东西是件新鲜事，而我们并没有真正讨论世界现在的真实样子以及人们的选择是什么。

But from my perspective, from the perspective of millions of other people, like what the hell is going on here? So, I want to talk about that because I think in a democratic society at least, at the end of the day, it shouldn't be up to a small group of people controlling AI companies. It shouldn't even in the fullness of time be up to Democratic leaders that are in power right now. It's up to people like the people watching this. I think that people just don't feel that way right now. It feels like everything's happening to them, us, it feels like there's just so many decisions that they're not part of. And part of why people often feel that way, if we take AI out of it for a sec, is we forget about the systems that already exist and we allow ourselves to be in a conversation that sounds something like should we regulate AI as though it's like a novel thing to regulate something and we're not having an actual conversation about what the world really looks like right now and what people's options are.

所以我准备了一些视觉资料，因为显然，我认为基本的思考方式是，对于任何触及消费者的东西，有些是在它们触及消费者之前就被检查的，并且持续检查，而有些则只在到达人们手中后才被检查，这取决于这个东西有多重要、有多风险等等。所以如果你想想具体的例子，核电站，NRC 在核电站运行前就批准，然后还有人在现场一直检查，这似乎，我们稍后会谈到，在某些方面似乎和 Daario 所要求的很接近。然后你有 FDA 在药物到达我们手中之前就批准，然后也持续检查。FAA 检查飞机也是一样，对吧？这些都是事先批准然后定期检查的。但还有其他高风险的东西，对吧？汽车。NHTSA 提供联邦指南。制造商证明他们符合这些指南。然后政府购买路上的汽车来检查。然后还有像补充剂这样的东西，实际上属于 FDA 的管辖范围，但它们是事后检查的。这些大多被涵盖在，如果人们在听比如 Jensen 的采访，他谈到责任和人们在受到伤害后事后起诉，那通常属于这个类别。你觉得这正确吗，就像你的第一个，我只是想在你深入探讨这如何适用于 AI 之前确认一下？

So I have some visuals because obviously so the basic way to think about this I think is for everything that touches a consumer there are things that are checked before they ever touch consumers and on an ongoing basis and there are things that are checked only after they reach people's hands and it depends on how important the thing is and how risky and all that. So if you think about the specific examples, nuclear power plants, the NRC approves nuclear power plants before they go live and then also has human beings at the sites checking them all the time, which seems, and we we'll get to this, seems pretty close to what Daario is asking for in some ways. Then you have the FDA approves drugs before they ever reach our hands and then also checks them on an ongoing basis. Same thing with the FAA checking airplanes, right? Like these are all things that are approved beforehand and then regularly checked. But there are other high-risisk things, right? Cars. The NHTSA offers federal guidelines. Manufacturers certify that they've met those guidelines. And then the government purchases cars off the road to check them. And then you have things like supplements that are actually under the purview of the FDA, but they're checked after the fact. Those are mostly covered by what if people are listening to you know the interviews with Jensen for example and he's talking about liability and like people suing after the fact after harm like that's generally this category. Does this feel right to you as like your first I just want to check before I dive into how this applies to AI?

是的，我觉得这大致正确。是的。

Yeah, I think this seems roughly right. Yeah.

好的。那么我们有，等等，这是有趣的部分。那么我们有所有的 AI 领袖。你在这里。这是我根据人们通过公开声明表达自己的位置所做的评估。我对 Daario 会说 AI 在哪里的理解是，他会说它在这附近的某个地方。他会特别说，他要求对模型进行现场持续检查，有真人在现场，这听起来有点像我们目前对核能所做的。然后你有像 Elon 和 Sam 这样的人加入进来说 Daario 是对的。我实际上会说 Zuck 在他说 AI 在哪里的问题上处于中间位置。我会说 Jensen 现在可能在这边。他基于他的公开声明，一般来说有一种观点是，如果它会伤害人，就不要发布。我们有激励措施。这就是责任，也有法律基础设施。

Okay. So then we have Wait, this is the fun part. So then we have all of our AI leaders. There you are. This is my assessment based on where people have expressed themselves via their public statements. My understanding of where Daario would say AI is is he would say it's somewhere over here. He would say specifically he's asked for inperson ongoing checks on models with physical human beings at the sites which sort of sounds like what we currently do for nuclear. And you have like Elon and Sam weighing in saying like Daria's right. I would actually say Zuck is somewhere in the middle in terms of where he's saying AI is. I would say Jensen is maybe over here right now. He's based on his public statements about generally there is a point of view that is like if it would hurt people don't ship it. We have incentives for that. That's what liability is like there is a legal infrastructure for that as well.

如果你的产品还没准备好发布，就不要发布产品。

If your product is not ready to ship don't ship the product.

所以我的问题是，你能帮我帮助人们思考这个问题吗？比如当他们听到关于如何监管 AI 的新闻时，你怎么想？你如何把自己放在这个混乱中？

So my question is, can you help me help people think about this? Like when when they're hearing news about how to regulate AI, how do you think about this? And how do you place yourself in this mess?

你知道，你说得完全正确。这太令人困惑了，因为从表面上看，每个人说的和每个人做的感觉如此疯狂地分歧。

You know, you said exactly right. It is so confusing because at face value, what everyone's saying and what everyone's doing feel so crazily divergent.

是的。

Yeah.

所以你有点处于这个世界中，我该相信他们说的吗？我该相信他们做的吗？人们到底站在哪里？他们到底相信什么？嗯，我也发现自己问这个问题。比如我觉得很有趣的是，你知道，那些倡导放缓前沿的人也在建湿实验室。我觉得那真的是一个很奇怪的因素汇合。所以我认为我的立场是，最终这项技术会变得越来越有能力。而且你知道，既在科学上非常令人兴奋，但也把这些所有问题带到前台的部分原因，就是 AI 移动得有多快。即使在过去一年，一年前大多数人还是手工编码。他们会进入代码编辑器写代码，现在人们大多只是和智能体对话。所以一切移动的速度相当惊人。嗯，我认为很多担忧被加剧，很多恐惧被加剧，是因为如果事情移动得这么快，一年后会在哪里？两年后会在哪里？政府和公众的反应时间线比 AI 进展的速度慢得多。所以等到一年、两年、三年或多少年后，当公众和政府真正决定他们想做什么时，谁知道 AI 那时会在哪里，会移动得多快。所以这是我认为能解释部分对话的事情之一。嗯，我认为另一件能解释部分情况的事情是所需的集体行动水平，以及各个 AI 领袖之间存在的普遍不信任，这已经被广泛报道，但一个主要的理由是，哦，我是负责任的，我会负责任，但那些人不负责任，无论那些人是另一个 AI 公司还是中国，或者谁知道什么妖怪，所以我不能放慢，否则那些不负责任的人会，谁知道他们会做什么。嗯，这个基本逻辑我认为非常令人困惑，因为我认为对大多数人来说，常识性的答案是，管其他人怎么样，你必须做你觉得对的事，对吧，你应该做符合你道德的事，但我认为这也是让很多事情复杂化的原因。所以我认为很多必要的事情几乎就是外交，这个词是对的，我们如何让这群人，他们彼此并不太信任，但都同意这是非常强大的技术，我们需要弄清楚如何着陆，如何确保我们负责任地开发它，并让事情顺利。

And so you're sort of in this world where do I believe what they say? Do I believe what they're doing? Like where do people actually sit? What do they actually believe? Um and I find myself asking this too. Like I think it was it's very funny that you know um the same people who are advocating for pacing the frontier also are building a wet lab. Like that's I think a really um weird uh confluence of of factors. So I think where where I stand is like ultimately um this technology will become more and more and more um capable. And uh you know part of what is both scientifically very exciting but also I think brings all these questions into the foreground is just how quickly AI has moved. Even over the past year like a year ago most people like coded roughly by hand. they would like go into the into the code editor and write code and now people are mostly just talking to agents. So the speed at which everything is moving is is quite staggering. Um and I think a lot of what the a lot of the concerns are are heightened and a lot of the fear is heightened by the fact that um like if things are moving this fast, where is this going to be in a year? Where's this going to be in two years? the government and the public react on a much slower time horizon than the pace of AI progress. And so by the time in one year or two years or three years or how many however many years forward when um the public and government are really deciding what they want to do, who knows like where AI is going to be at that point and how fast it's going to move. So that's one of the things I think explains some of the dialogue. Um I think the other thing that kind of explains some of this is the level of collection collective action necessary and the level of general mistrust um that exists between the various AI leaders towards one another and this is like well covered but one major justification for folks is that like oh I am responsible and I would be responsible but those guys aren't responsible whether those guys are you know another AI company or China or you know who knows what boogeyman so I can't slow down otherwise those guys who are irresponsible are going to you know who knows what they're going to do. Um and that basic logic I think is very confusing because I think for most people um uh the kind of like common sense answer is well who cares what the other guys are like you have to you have to do what feels good to you right and like you should do what's within your ethics but I think that's also what complicates a lot of this so I think a lot of what is necessary is like it's almost diplomacy is the right word like how do we get this group of people who all who don't really trust each other that much, but all agree that like this is really powerful technology and we need to we need to figure out how to land it and how to how to make sure that we develop it responsibly and that things go well.

我们怎么让这一切运转起来？我认为我们正处在疯狂程度的顶峰，因为所有这些问题都在公众面前被摊开。我们都能看出没人信任彼此，也能看出人们非常担忧，也许比 AI 本身真正值得担忧的时间点还早了一点。所以我最终认为，最重要的是让所有人就某些共同的基础和底线达成一致，然后制定某种流程，让我们都能重新评估并持续审视，说嘿，我们都同意现在需要做这件事，现在都需要做那件事，我们都看到模型在某些方面变得强大，让我们一起看看这个。而且我真心相信，这种集体对话不仅在美国的 AI 领袖之间可能实现，在国际上也可能实现。我认为我们都有共同利益，确保这项技术不会走向可怕的错误。

Like how do we make all this work? And we're at the moment of peak insanity, I think, for that because it's like in the public all these problems are being aired out. We can all tell that nobody trusts each other and we can tell that people are being really worried, maybe a little bit before the AIs themselves are getting to the point where they're deserving of that worry. And so I ultimately think the most important thing is to get everyone to agree to some common foundation and base and then develop some process by which we can all re-evaluate and consistently look at it and say hey we all agree actually that we need to be doing this now and we all need to be doing this other thing now and we all see like the models are getting powerful in certain ways like let's all look at this together and I actually genuinely believe, for what it's worth, that having these collective dialogues is not only going to be possible among the AI leaders within America but also going to be possible internationally. I think we all have a shared interest in making sure this technology does not go horribly wrong.

是的。那么显而易见的下一个问题就是中国。你谈到过这是一个重大关切。这些 AI 公司高层互不信任，但他们常常说：“我们必须尽可能快，因为万一中国先到了呢？”比如，特德·克鲁兹公开说：“我们宁愿要美国的杀人机器人，也不要中国的杀人机器人。”

Yeah. I mean the obvious next question is the China of it all. I mean you've talked about how this is a big concern. People don't trust each other at the top of these AI companies, but they all often say, "We have to go as fast as possible because what if China gets there first?" I mean, you Ted Cruz is out there saying like, "We would rather have US killer robots than Chinese killer robots."

如果它们要成为杀人机器人，我宁愿它们是美国的杀人机器人，而不是中国的杀人机器人，

If they're going to be killer robots, I'd rather they be American killer robots and not Chinese killer robots,

我觉得这吓到了很多人。而且说得委婉一点，是的，这是美国法律现有美国法律的一个光谱。这不包括任何全球动态，更不用说长期合作了。考虑到中国因素，你如何评估这个光谱？那到底意味着什么？我该如何思考中国与这种动态的关系？

which I think scares a lot of people. Um and also to say it mildly um yeah this is a spectrum of American law existing American law. This does not include any global dynamics not to mention like long-term collaboration. How do you evaluate this spectrum given the China of it all and and what what does that even mean? Like how should I think about China with respect to this dynamic?

我个人甚至改变了自己在这个问题上的看法。过去我非常担心中国，非常担心那种动态，我读过一些中国官员过去的演讲，谈到他们如何将 AI 视为超越西方的机会，读起来确实令人担忧。但随着时间的推移，我认为我们把他们当成了太大的妖魔，而实际上他们会看到和我们一样的事情：这是强大的技术，我们作为人类需要把它做好。在确保 AI 顺利发展方面会有很多共同目标，我认为确实有可能达成某种形式的国际合作。我认为中国政府像任何政府一样，有兴趣确保社会稳定，确保他们不失去对技术的控制，确保总体顺利，不会出现灾难性的生存风险，所以有相当多的共同点，我认为这将使每个人都能理性思考，最终做出正确的决定。即使现在看起来完全疯狂，在某些方面感觉不可能，我真的相信，随着人们越来越多地看到和了解这项技术，人们会做出正确的选择。

I've actually like personally even changed my own opinion on this matter. Like I think in the past I was very worried about China and I was very worried about what that dynamic looked like and I had read these like speeches that were given by PRC officials in the past that talked about how they viewed AI as like this opportunity to leapfrog the West and you know these are concerning things when you read them. But as time has gone on, I think that we're paying them not to be too much of a boogeyman when in reality they are going to see the same things that we see in terms of this is powerful technology and we as a humanity need to get this right. There's going to be a lot of shared goals in making sure that AI goes well and I think it really is possible to arrive at some form of international cooperation. I think that the Chinese government like any government has an interest in ensuring that there's stability in their society and that they don't lose control of the technology and that overall things go well and we don't have catastrophic existential risks and so there's quite a bit of shared ground that I think will enable everybody to be rational and think rationally and ultimately make the right decisions. And even though it seems totally insane right now and it seems like in some ways that feels impossible, I really have faith actually that as people see more and more and learn more and more about the technology, people make the right calls.

你最近成为了美国那场对话中最重要的 20 人之一。过去几年你在 AI 领域非常重要。我想人们记得你来到 Meta 时登上头条。这对你来说不是全新的，但过去几周，你设计并发布了一款产品，使用人数比 ChatGPT 最初几周还多。感觉你被推到了这场对话的一个新位置，坦率地说，人们会问你更多问题，比如 Alex，你到底在这个光谱的哪个位置？你希望看到什么，因为你是负责人之一。你觉得你已经得出结论了吗？

You have recently become one of the 20 most important people in that conversation in the United States. You've been very important in AI in the last couple years. I mean, I think people remember seeing you in the headlines when you came to Meta. I think this, you know, isn't brand new to you, but in the last couple weeks, you have designed and released a product that is being used by more people than ChatGPT was in its first couple weeks. It feels like you have been thrust into a new kind of part of this conversation where frankly people will be asking you more questions like where exactly, Alex, do you fall on this spectrum? like what would you want to see because you're one of the people responsible. Do you feel like you've come to those conclusions yet?

是的，我认为任何从事这项技术的人，这些事我已经思考了很多年。我真正思考我想做什么、想实现什么以及如何塑造这场对话的方式，是让每个人都能进行共同的对话，然后集体到达正确的地方。我不认为在对话的这个阶段，我说“哦，我相信这个，你们都该听我的，因为我构建了 Muse 之类的”有多大帮助。我们已经有很多人，他们都相信略有不同的事情。最终，这是一项我们需要集体行动的技术，每个人都需要同意，我们都需要到达正确的地方。所以我真正关心的是，我们如何促进并在公开和私下进行相关对话，使我们能够达成一致路径。对我来说非常奇怪的一件事是，我认识很多这些人，我见过他们。他们都很聪明。但某些个人恩怨实际上在定义人们说什么、做什么以及如何开发这项非常重要的技术，这对我来说太疯狂了。在某种程度上，这是一个非常人性化的故事：恩怨或者只是我真的很不信任另一个人，可以滚雪球变成文明级别的问题。但我不认为这对我们任何人有利。我认为最终我们必须把分歧放在一边，把历史放在一边，一起弄清楚如何解决这个问题。

Yeah, I think like anyone who works on this technology, these are things that I've been thinking about for years and years. Truly the way I think about what I want to do and what I want to accomplish and how I want to shape this conversation is to enable everybody to have a common dialogue and then get to the right place collectively. Like I don't think at this point in the conversation it's that helpful for me to say, "Oh, I believe this and you all should listen to me because I built Muse or whatever." We already have a bunch of people. They all believe slightly different things. Ultimately this is a technology that we need collective action on that everyone needs to agree on that we all need to get to the right place. So what I really care about is how do we facilitate and have the relevant dialogues in public and private that enable us to get to the point where we all agree on a path. And one of the very strange things for me is I know a lot of these people and I've met them. They're all brilliant. But the degree to which certain personal animosities are actually defining what people say and do and how they're developing this really important technology is insane to me. It is in some ways a very human story that feuds or just the fact that I really don't trust this other person can snowball into this civilizational level problem. But I don't think that serves any of us. I think ultimately we have to all put our differences aside, put our histories aside and just figure out how we solve this together.

听起来很笼统，但这是一个相当具体的要求。我的意思是，存在阻止这种情况的限制。目前有反垄断法规阻止很多这类对话发生。而 Dario 提出的众多提案中的一项是豁免那些反垄断法，以允许这类对话。

It sounds very general but that's a fairly specific ask. I mean there are constraints that prevent that. Right now there are antitrust regulations that prevent a lot of those conversations from happening. And one of the proposals, one piece of many proposals that Dario put forward is waivers from those antitrust laws that would allow those kinds of conversations.

我的意思是，如果有什么秘密对话正在进行，请告诉我我错了，但我不认为首先在美国有一个达成共识的团体在进行这种对话，然后还有，我们甚至如何开始与中国进行这些对话？但实际上，我不知道的是什么？这是否正在某个私人房间里发生？还是说我们仍处于这个过程的开始，需要弄清楚如何去做？

I mean, tell me if I'm wrong if there are secret conversations happening, but I don't think there is first the agreed upon group in the United States that's having that conversation and then also like how do we even begin to have those conversations with China? But actually, what don't I know? Is that something that's happening in, you know, some private room somewhere? Or is it that like we're still at the beginning of that and we need to figure out how to do it?

嗯，我不认为现在正在发生。在白宫有过这样的对话，并且提出了这些协议，这是真正的一步，就像我认为让每个人都同意某事是非常重大的，每个人都同意某事，我认为那一件事基本上是说嘿，我们将要，每家公司都将制定并发展他们围绕开发这项技术的内部控制和原则，然后会有多层验证，从内部团队验证到外部审计师验证他们遵循这些控制，再到董事会级别的委员会确保你遵循这些控制。就像我认为这是一个伟大的步骤，因为从很多方面来说，现在需要的很多只是透明度。就像我们只需要确保人们正在安全和理性地进行这件事，我认为在某些方面，人们甚至不一定信任这一点。作为一个行业，我们有很多信任建设要做。我是一个乐观主义者。我认为你是一个乐观主义者。就像我认为我们都是乐观主义者。我真的认为这些对话会自然发生。我提到的最后一个协议，那是在白宫。政府帮助促进了这一点。所以我认为我们会看到更多这样的情况。这将是颠簸的。会有完全混乱和混沌的时刻，就像到底发生了什么，但我也看到了进步。

Um, I don't think it's happening right now. At the White House there was this dialogue um and these accords put forth which are a real step like I think there's um getting everyone to agree to something is like that's really big everyone agrees on something and I think that one that thing which is basically saying hey we're going to every company is going to set out and develop what their internal controls and principles around developing this technology look like and then there's going to be multiple layers of verification on that from internal teams that verify to uh external auditors that validate that they're following those controls to board level committees that ensure that you're following those those um controls. Like I think that's a great step um cuz in in many ways a lot of what's needed right now is just transparency. Like we just need to ensure that you know people are going about this sely and rationally and I think in some ways like people don't even trust that necessarily. As an industry, we have a lot of trust building to do. I am an optimist. I think you're an optimist. Like I think we're all optimists. Like I really do think these conversations will naturally happen. This last accord that I referenced, like that was in the White House. The government helped facilitate that. And so I think we're going to be seeing a lot more of that. It's going to be bumpy. There will be moments of total confusion and chaos and like what the heck is going on, but I see progress, too.

我很高兴我们在这次对话中花了很多时间谈论安全和监管。现在我们如何回到对未来的乐观愿景？我认为很多人在长时间谈论信任和安全后自然会问的问题是，为什么要费心做这个？这是 SNL 小品中的时刻，你知道，如果有 10% 的机会某件事会杀死你，但它可能会让你的生活更好，你会不会做，另一个人说不会，我不会

I'm glad that we've spent a lot of this conversation talking about safety and regulation. How do we bring it back now to the optimistic vision of the future? I think the question a lot of people will naturally have after a long conversation about trust and safety is why bother doing this at all? This is the moment in the SNL sketch where it's uh you know if there's a 10% chance that something will kill you but it'll probably make your life better like wouldn't you do it and the other guy's like no I wouldn't

不

no

真的

really

你能帮助人们回答这个问题吗，比如为什么要做这个

can you help people answer that question like why do this

我一直觉得有点悲剧的是，我认为几乎每个人在小时候都有大梦想和大想法，他们想改变世界，想拯救地球，想写一本书或制作百老汇音乐剧或拍电影。就像他们真的有大量的希望，然后由于某种原因，当人们完成学业并找到工作时，那种火焰就熄灭了，人们真的停止梦想大，停止有大胆的想法，停止希望大事和很多改变，尤其是今天，我认为很多年轻人变得非常愤世嫉俗，他们说哦世界很糟糕，确实如此，我不知道为什么它在很多方面如此疯狂，但我想这就是它的方式。他们失去了对自己有能力促成这种改变的信心。我真的认为 AI 的承诺之一，尤其是像 Muse 这样的东西，你把 AI 给世界上的每一个人，就是这不会发生。人们可以拥有大想法、大梦想、愿望和目标，他们可以在小时候拥有，在青少年时拥有，在成年后继续拥有。而且，你知道，一个 AI 超级智能助手在那里帮助他们实现这一切。这可以是从大愿望和大梦想，到小愿望，比如嘿，我只想多陪陪孩子，或者我只想多陪陪父母，或者我只想吃得健康点。一直到，嘿，我有一个疯狂的想法，比如我想开一个博物馆。我的伴侣真的想做一个冰博物馆。我觉得这太搞笑了。但是

something that I have always felt was a bit tragic was that I think um almost everyone when they're kids they have big dreams and big ideas and they want to change the world and they want to save the planet and they want to, you know, uh, write a book or make a Broadway musical or make a movie. like they have really they have tons of hope and then for some reason or another by the time that people finish school and they get a job that like flame kind of dies and people really um they stop dreaming big and they stop having big ideas and hoping for big things and hoping for lots of change and especially today like I think a lot of young people get really cynical and they're like oh the world is horrible and it is and I don't know why it's so insane in so many ways, but I guess this is just the way it is. They lose faith that they have an ability to enable that change. And I genuinely think one of the promises of AI and especially something like Muse where you give AI to every single person in the world is that this doesn't happen. that people can have big ideas and big dreams and wants and and goals that they can have as little kids, they can have when they're teenagers, they can keep having through adulthood. And that, you know, an AI super intelligent sidekick is there to help them just make all that happen. And that can be everything from like big wants and big dreams um to like small wants like, hey, I just want to spend more time with my kids or I just want to spend more time with my parents or I just want to like eat healthier. Um all the way up to, hey, I have this like crazy idea for um like a a museum that I want to start. Like my my partner really wants to make a museum of ice. I think it's like hilarious. But um

冰太疯狂了。有一个关于寒冷的演变的惊人故事，关于我们如何保持东西冷，从冰开始。那很酷。我会去博物馆。

ice is crazy. There's a whole amazing story about the evolution of cold of how we keep things cold that starts with ice. That's pretty cool. I go to the museum.

没错。人们有这些想法，然后他们就是做不到，你知道，在某些方面年轻人是对的。就像今天的世界只是一个由路障、问题、挑战和后勤之类的东西组成的迷宫，你知道，那些阻碍你实现这些事情的东西。而今天，在小的方面，我看到 Muse 让这个问题变得更好。就像我看到人们用它来节省时间、省钱，把小事从他们的盘子里拿走，给他们越来越多的能动性、越来越多的自由、越来越多的能力，把时间花在他们想做的事情上。我认为 AI 的承诺之一，它变得越来越好，不仅更聪明，而且更对齐、更值得信赖，就是你可以给它越来越大的想法。你可以说，嘿，我想开一个冰博物馆。告诉我怎么做。它会告诉你这些是你可以租的建筑，这是你可以联系的资金来源，他们可能对冰充满热情，这是你应该如何考虑策划你的展览，你知道，那可能是什么样子。我认为对于像你或我这样经历过这种旅程的人，有大想法，然后穿过所有疯狂的迷宫来实现它。这真的很神奇，很特别，我认为这是，你知道，我认为人类有梦想然后实现它们的故事是最，是我能想象到的一些最美丽的事情。我真的认为有了 AI，就像每个人都有一个小 Muse 助手，更多的人可以体验这个。我希望每个人，真的像世界上所有 80 亿人，都能体验这个并有能力做到。那只是一个非常特别的世界。我认为如果我们能做到这一点，我认为更多的人能够从生活中得到他们想要的，过上更充实的生活，这将是神奇和令人兴奋的。这就是我真正认为对人们的承诺。我认为部分令人困惑的是，你知道，很多时候我们在 AI 领域说，哦 AI 会治愈癌症，我认为每个人都希望癌症被治愈，但每个人都有一个他们强烈想要的东西，他们想要的东西甚至比讽刺地说，治愈癌症还要多。

Exactly. And and people have these ideas and then they just can't you know in some ways young people are right. Like the world today is just kind of a maze of like roadblocks and issues and challenges and logistical um you know uh things that get in your way to make those things happen. And in a small way today, I'm seeing Muse make this problem better. Like I'm seeing people using it to save time, save money, like take the little stuff off their plate, and just give them more and more agency, more and more freedom, more and more of an ability to spend their time on the things they want to do. And I think one of the promises of AI that gets better and better and better, gets not only smarter, but also more aligned and more trustworthy, is that you can give it bigger and bigger ideas. And you can say, hey, I want to start a museum of ice. Like, tell me how to go about doing this. and it'll tell you these are the buildings you can lease and this is who you can contact for funding who might be passionate about ice and um this is like how you should think about curating your exhibit and you know uh what what that might look like and I think for some people like you or me who have experienced this this like journey of having big ideas and then like going through all the crazy maze of things to make that happen. It is like really magical and it's really special and I think it's something that um you know I think human stories of people who have a dream and then accomplish those are the most like are some of just the most beautiful things that I can possibly imagine. And I truly think with AI and like everyone with a little Muse sidekick like more people can experience this. I my hope is that everyone literally like all eight billion people in the world experience that and have the ability to do it. And that is just like a very special world to live in. I think if we can make that happen and I think more people will be able to get what they want out of their lives and and live more fulfilled lives and it'll be magical and exciting. And that's what I really think the promise is for for people. And I think part of what is so confusing is like um you know often times people in AI we say like oh AI will cure cancer and I think everyone wants cancer cured but every individual person has something they want intensely and they want even more than you know ironically speaking a cure for cancer.

所以我觉得神奇之处就在于，我们谈论的这个东西——个人超级智能——它会帮助你去实现你想要的事、你梦想的事。而这在你一生中会不断变化，你将能够实现许多梦想和许多愿望。我认为这就是 AI 的承诺。

And so I think that's part of the magic, is that we talk about this thing, personal superintelligence, like it will help you at the thing that you want and the thing that you dream of. And that will change over your life, and you will be able to accomplish many dreams and many wants. And I think that's the promise of AI.

我觉得那个梦想里包含着一个信念，就是人们确实怀有宏大的梦想，而这些宏大的梦想往往能帮助到其他人。所以当我们听到有人说 AI 会治愈癌症时，有时也许他们说的是你给 AI 一个提示，它就想出了治愈癌症的方法，但更可能得多的情况是，AI 促进了数百万癌症医生的个人抱负，我们一起加速了攻克癌症的进程。

That dream has within it a belief, I think, which is that people do actually have big dreams and that those big dreams will more often than not help other people. So when we hear people say AI will cure cancer, sometimes perhaps they're talking about you prompt an AI and it thinks up the cure for cancer, but much, much more likely it's the AI facilitates the individual ambitions of millions of cancer doctors and together we accelerate the progress toward cancer.

这让我想到，当我听到人们谈论 Muse 时说，人们要 AI 助手干什么？人们没有那些梦想。这是对他人多么狭隘的看法。我不知道那些梦想是什么，而且也不该由你或我来评判哪个梦想比另一个更有价值，就像你说的，无论大小。

It's making me think, you know, when I hear people talk about Muse and say, like, what do people need an AI assistant for? People don't have those dreams. Like, what a limited view of other people. You know, I don't know what those are, and it's not up to you or me to say which dream is more worthy than the other, to your point, big or small.

我对人的看法是，他们会有灵光一现的时刻，在某个时候他们会意识到，等等，我真的很想做这件事。我希望每个拥有这个东西的人都能告诉他们的 Muse，嘿，我有个大想法。而他们的 Muse 就会在幕后悄悄地帮他们弄清楚如何实现它。我觉得这其中一部分也是所谓的煽风点火。我认为对很多人来说，他们会有一个小小的愿望，然后它实现了，接着他们会梦想得更大一点，然后它真的实现了，他们会梦想得更大一点，它又真的实现了。随着这个过程继续，人们会逐渐意识到，AI 可以远不止于此，Muse 可以远不止于此，他们自己也能做到远不止于此。这不会一夜之间发生。这需要时间，每个人都必须自己去体验，我真心希望每个人都能体验到。但我认为，经过几代人，我觉得这可以变得绝对神奇。

My view on people is like, they will have moments of inspiration where at some point or another they will realize, like, wait, I really want to do this thing. I want everyone who has one of these to be able to tell their Muse, hey, I have this big idea. And their Muse will just, like, secretly behind the scenes help them figure out how to make it happen. And I think part of this is also fanning the flame, so to speak. I think for a lot of people, they will have, like, a small want, and then it'll happen, and then they'll dream a little bit bigger, and then it'll actually happen, and they'll dream a little bit bigger, and it'll actually happen. And as that goes on, people will sort of realize that AI can be so much more, that Muse can be so much more, that they can do so much more. This won't happen overnight. This is something that will take time, and everyone will have to experience on their own, and I really hope everyone experiences it. But I think over, like, generations, I think it can be absolutely magical.

你知道，我觉得尤其是你在媒体上被谈论的方式，你会自然而然地变成自己的一个漫画式形象。世界上存在着一个关于你的角色。而这帮助我更好地了解了你，因为在你的一些回答里似乎有一个共同的主题，那就是 Muse 是一种促进他人梦想的方式，因为你对人、对人的善良、对他们改善世界的能力、以及他们想法的价值抱有信念。看起来你在某些方面引导着 Muse，但你并没有说这是用于编程的，或者这是用于某个特定任务的。你在观察并促进。当我问你在这个问题上持什么立场时，你的回答是，我感兴趣的是群体的集体智慧。我感兴趣的是促进或参与其他人之间的对话，我们世界各地有很多聪明人，我不觉得说这就是我对这件事究竟该如何监管的确切看法会有多大用处。我认为最好的结果来自许多不同的人带着他们各自许多想法和愿望的群体对话。你同意我刚才说的吗？

You know, I think especially the way that you're in the press and talked about, and you kind of naturally become sort of a caricature of yourself. There's like the character of you out there in the world. And this has helped me get to know you better, because there seems like a common theme in some of your answers, which is Muse is a way to facilitate other people's dreams, because you have a belief in people and their goodness and their ability to improve the world and just the sort of value of their ideas. And it seems like you are directing Muse in some ways, but you're not saying this is for coding or this is for X specific task. You're watching and facilitating. And when I asked you where you fall on this, your answer was, I'm interested in the collective wisdom of the group. I'm interested in facilitating or being part of a conversation between other people, that we have a lot of smart people all around the world, and I don't feel like it would be that useful to say here's exactly my opinion on where exactly this should be regulated. I see the best outcome as coming from a group conversation of many different people with their many thoughts and wants. Do you agree with what I've just said?

是的，我认为因为关于 AI 有太多重大问题，而且它是那种每个人都有观点、也应该有观点的事情。我觉得民主制度、民主国家和广义上的民主之所以了不起，部分就在于每个人的想法都很重要，我们必须弄清楚如何把大家想法的这团复杂、杂乱的东西引导到正确的事情发生上。另外一点是，阻碍 AI 发展的部分原因，也是我认为 AI 问题的一部分，就在于它几乎让人感觉构建 AI 的人在居高临下地对别人说话。就像，嘿，我在构建你所知道的最聪明的东西。它会比你聪明得多，而我确切知道我们要拿它做什么。我认为人们理所当然地会说，等等，什么？你说的这些跟我有什么关系？这对我又有什么好处？然后你说，哦，它能治愈癌症。我说，好吧，但那又怎样？我该相信你吗？

Yeah, I think because there are so many big questions about AI, and it is like something that everyone has a point of view on and should have a point of view on. And part of I think what's amazing about democratic institutions and democratic countries and democracy broadly speaking is that it matters what everyone thinks, and we have to figure out how we take this complex, jumbled mess of what everyone thinks and channel that into the right thing happening. And another thing is, part of what has held AI back, and I think part of the problem of AI, is how it almost feels like the people building AI are talking down on other people. It's like, hey, I am building the smartest thing you've ever known. It's going to be way smarter than you, and I know exactly what we're going to do with it. And I think rightfully people are like, wait, what? Like, what does anything you just said have to do with me? And how is this even good for me? And then you're like, oh, it'll cure cancer. I'm like, okay, but what else? Like, do I believe you?

是的。

Yeah.

所以我认为，尤其是考虑到我们当下所处的时刻，就像你说的，人们对 AI 有多么恐惧，有多么不确定，所说的一切有多么令人困惑，我觉得最重要的事情是，首先，让每个人都去体验它。让每个人都真正理解这种热度——找不到更好的词了——感受 AI 智能体能为你做什么，能如何让你的生活变得更好。这会给你一个视角，让你明白，哦，这是一项技术，是的，它肯定有风险，有大量风险，别误会我，但它也确实很棒，我喜欢它，我想确保至少它的某种形式能延续下去，让我能继续使用。一直到那些重大问题和大对话，比如，我们如何让每个人都达到一个我们都认同、并能找到前进道路的状态？我认为我有足够的谦逊，知道这些人都是大人物、重要人物。鉴于他们的观点，我不知道我还能补充多少。但我只是想确保我们去促进——这个词一直在我脑海里浮现——我们如何快速行动，共同形成有依据的观点，然后进入一个我们就某事达成一致的进程。

And so I think that especially given the moment that we're in, and to your point, like how much fear there is about AI and how much uncertainty there is and how confusing everything that's being said is, I think the most important things are, like, first, let's have everyone experience it. Like, let's have everyone actually understand the hype, for lack of a better term, feel what AI agents can do for you and can do to make your life better. And that'll give you perspective as to, oh, this is a technology that, yeah, it's definitely risky, and there's tons of risk, don't get me wrong, but it also is really great, and I like it, and I want to make sure at least some form of it continues and I can continue using it. All the way to, you know, these big questions and these big dialogues, like, how do we just get everyone to a place where we all agree and we can find a path forward? And I think I have enough humility that, like, these guys are, you know, they're big important people. Like, I don't know if I have much to add, you know, given their points of view, but I just want to make sure that we facilitate — is really the word that keeps coming to my mind — like, how do we move quickly to all collectively come to informed opinions and then come to a process where we agree on something.

这是我在这些长篇对话结尾都会问每个人的问题，那就是，当我想到我去世之后，当人们说我是一个非常好的家庭成员、她爱她的丈夫、她爱她的朋友之后，我希望他们谈论这个节目的使命。我希望他们谈论我如何尽了最大努力去帮助人们看到未来可能变好的种种方式，因为我真的相信——看起来你也一样——相信其他人，相信他们的视角，当他们看到一个自己相信的未来愿景时，他们就会真正去努力帮助它实现。这就是我投入全部工作时间在做的事情的意义。你显然才刚刚起步。你已经产生了如此巨大的影响。你还有很长的路要走。我很好奇，当你想到你离世后希望被人记住的方式时，你希望人们说些什么？

This is the question I ask everyone at the end of these huge conversations, which is, when I think about after I'm dead and after people say that she was a really great family member and she loved her husband and she loved her friends, I would want them to talk about the mission of this show. I would want them to talk about how I tried my best to help people see ways the future could go well, because I really believe, as it seems you do, in other people, in their perspective, and when they see a vision of a future they believe in, that they're going to actually try and help make it happen. Like, that's the point of what I spend all of my working hours doing. You're obviously just starting out. You've already had this huge impact. You have a lot farther to go. I'm curious, when you think about how you want to be remembered after you're gone, what do you want people to say?

嗯，首先，多么美好，多么美好的一个想法和概念。

Um, first of all, what a nice, what a nice idea and concept.

对我来说，我希望人们会说，我构建的东西和我所贡献的东西，帮助每个人从他们想要的生活中获得更多。

I think for me, I would want people to say that the things I built and the things I contributed to helped everyone get more out of what they wanted out of their lives.

太棒了。谢谢你接受这次访谈。

Beautiful. Thanks for doing this.

谢谢。

Thank you.