Sam Altman 谈 AI 创造力、知识产权与工作未来

Sam，欢迎来到 TED。非常感谢你的到来。

Sam, welcome to TED. Thank you so much for coming.

谢谢。这是我的荣幸。

Thank you. It's an honor.

你的公司感觉每隔一周就在发布疯狂的新模型。我玩过其中几个，想给你看看我玩的东西。这是 Sora，图像和视频生成器。我问了 Sora 这个问题：当你在 TED 分享一些震撼性的爆料时，会是什么样子？你想看看它怎么想象的吗？（笑声）我觉得还不错，对吧？你给打几分？所有手都是五根手指。

Your company has been releasing crazy insane new models pretty much every other week it feels like. I've been playing with a couple of them. I'd like to show you what I've been playing. So, Sora, this is the image and video generator. I asked Sora this: What will it look like when you share some shocking revelations here at TED? You want to see how it imagined it, you know? (Laughter) I mean, not bad, right? How would you grade that? Five fingers on all hands.

非常接近我穿的衣服，挺好的。

Very close to what I'm wearing, you know, it's good.

我从没见过你这么活跃。

I've never seen you quite that animated.

不，我不是那么活跃的人。

No, I'm not that animated of a person.

那可能是个 B+。但这个真的让我震惊。当我让它画一个图来展示智能和意识的区别时，它就这样做了。我的意思是，这很简单，但太不可思议了。是什么样的过程让它能做到——这显然不只是图像生成，它连接到了你整体模型的核心智能。

So maybe a B-plus. But this one genuinely astounded me. When I asked it to come up with a diagram that shows the difference between intelligence and consciousness. Like how would you do that? This is what it did. I mean, this is so simple, but it's incredible. What is the kind of process that would allow -- like this is clearly not just image generation. It's linking into the core intelligences that your overall model has.

是的，新的图像生成模型是 GPT-4o 的一部分，所以它拥有所有的智能。我认为这也是它能做到这些人们真正喜欢的事情的原因之一。

Yeah, the new image generation model is part of GPT-4o, so it's got all of the intelligence in there. And I think that's one of the reasons it's been able to do these things that people really love.

我的意思是，如果我是一个管理顾问，在玩这些东西，我会想，哦，我的未来会是什么样子？

I mean, if I'm a management consultant and I'm playing with some of this stuff, I'm thinking, uh oh, what does my future look like?

我认为你可以有两种看法。你可以说，天哪，它做我所有的工作，我会怎么样？或者你可以说，就像历史上每一次技术革命一样，现在有了这个新工具，我可以做更多的事情。我能做什么？确实，对某个特定岗位的人的期望会提高，但能力也会大幅提升，我认为很容易就能达到那个水平。

I mean, I think there are sort of two views you can take. You can say, oh, man, it's doing everything I do. What's going to happen to me? Or you can say, like through every other technological revolution in history, OK, now there's this new tool. I can do a lot more. What am I going to be able to do? It is true that the expectation of what we’ll have for someone in a particular job increases, but the capabilities will increase so dramatically that I think it will be easy to rise to that occasion.

这个也让我印象深刻。我让它想象查理·布朗认为自己是 AI。它生成了这个。我觉得这其实相当深刻。你觉得呢？（笑）我的意思是，一些新模型的写作质量，不仅在这里，而且在细节上，真的达到了新水平。

So this impressed me too. I asked it to imagine Charlie Brown as thinking of himself as an AI. It came up with this. I thought this was actually rather profound. What do you think? (Laughs) I mean, the writing quality of some of the new models, not just here, but in detail, is really going to a new level.

这是一个不可思议的元回答，但我们真的无法知道它是在思考，还是只是在训练集中多次看到过。当然，如果你分辨不出区别，你会在意多少呢？

I mean, this is an incredible meta answer, but there's really no way to know if it is thinking that or it just saw that a lot of times in the training set. And of course like if you can’t tell the difference, how much do you care?

这真的很有趣。我们不知道。但这不是……乍一看这像是知识产权盗窃。你们和《花生漫画》版权方没有协议吧？（掌声）你们想鼓掌就鼓掌吧，尽情享受。（笑声和低语）我想说的是，人类的创造精神是非常重要的，我们想构建提升它的工具，让新人能创作出更好的艺术、更好的内容、写更好的小说，让我们都能享受。我深信人类将处于中心。我也相信我们可能需要围绕创意产出的经济学找出某种新模式。我认为人们长期以来一直在借鉴他人的创造力，汲取灵感。但随着创造力的获取变得极其民主化，人们不断在彼此的想法上构建，我认为有令人难以置信的新商业模式，我们和其他人都很兴奋去探索。具体会是什么样子，我不确定。显然，有些明确的东西，比如你不能复制别人的作品。但你能汲取多少灵感？如果你说，我想以这七个人的风格生成艺术，他们都同意了，你如何分配钱给每个人？这些都是大问题。但历史上每一次，我们都把更好、更强大的技术交到创作者手中。我认为我们集体获得了更好的创意产出，人们做出了更惊人的东西。

So that's really interesting. We don't know. Isn't there though ... like at first glance this looks like IP theft. Like you guys don’t have a deal with the “Peanuts” estate? (Applause) You can clap about that all you want, enjoy. (Laughter and murmuring) I will say that I think the creative spirit of humanity is an incredibly important thing, and we want to build tools that lift that up, that make it so that new people can create better art, better content, write better novels that we all enjoy. I believe very deeply that humans will be at the center of that. I also believe that we probably do need to figure out some sort of new model around the economics of creative output. I think people have been building on the creativity of others for a long time. People take inspiration for a long time. But as the access to creativity gets incredibly democratized and people are building off of each other's ideas all the time, I think there are incredible new business models that we and others are excited to explore. Exactly what that's going to look like, I'm not sure. Clearly, there’s some cut and dry stuff, like you can’t copy someone else’s work. But how much inspiration can you take? If you say, I want to generate art in the style of these seven people, all of whom have consented to that, how do you, like divvy up how much money goes to each one? These are like big questions. But every time throughout history we have put better and more powerful technology in the hands of creators. I think we collectively get better creative output and people do just more amazing stuff.

更大的问题是当他们没有同意的时候。在我们的开场环节，Carole Cadwalladr 展示了，你知道，“ChatGPT 以 Carole Cadwalladr 的风格发表演讲”，果然，它发表的演讲不如她本人的好，但相当令人印象深刻。她说，“好吧，这很棒，但我没有同意这样做。”我们该如何处理这个问题？难道没有办法吗？应该只允许同意的人吗？或者不应该有一种模式，规定提示中任何被点名的人，如果其作品被使用，他们应该得到一些回报？

An even bigger question is when they haven't consented to it. In our opening session, Carole Cadwalladr, showed, you know, "ChatGPT give a talk in the style of Carole Cadwalladr" and sure enough, it gave a talk that wasn't quite as good as the talk she gave, but it was pretty impressive. And she said, "OK, it's great, but I did not consent to this." How are we going to navigate this? Like isn’t there a way, should it just be people who’ve consented? Or shouldn’t there be a model that somehow says that any named individual in a prompt whose work is then used, they should get something for that?

目前，如果你用我们的图像生成工具说，我想要某位在世艺术家的风格，它不会做。但如果你说，我想要某种特定氛围、工作室或艺术运动的风格，它会做。显然，如果你输出一首歌，像是某首歌的复制品，它也不会做。这条线应该划在哪里，人们如何说这太过分了，我们之前通过版权法和合理使用原则已经解决了。同样，我认为在 AI 的世界里，我们会找到一种新模式。

So right now, if you use our image-gen thing and say, I want something in the style of a living artist, it won't do that. But if you say I want it in the style of this particular like kind of vibe, or this studio or this art movement or whatever, it will. And obviously if you’re like, you know, output a song that is like a copy of a song, it won't do that. The question of like where that line should be and how people say like, this is too much, we sorted that out before with copyright law and kind of what fair use looks like. Again, I think in the world of AI, there will be a new model that we figure out.

从角度来看，我的意思是，创意人士现在是关于 AI 最愤怒或最害怕的一群人。感觉你的作品被偷走、未来被偷走，与感觉你的作品被放大、可以被放大，这两种感觉截然不同。如果我们能转向第二种感觉，我认为那会真正改变整个人类对这一切的接受程度。

From the point of view, I mean, creative people are some of the angriest people right now or the most scared people about AI. And the difference between feeling your work is being stolen from you and your future is being stolen from you, and feeling your work is being amplified and can be amplified, those are such different feelings. And if we could shift to the other one, to the second one, I think that really changes how much humanity as a whole embraces all this.

嗯，再次，我会说一些创意人士非常不满。一些创意人士说，“这是有史以来最棒的工具，我在做不可思议的新作品。”但你知道，这确实是一个变化。我对那些说“我希望这个变化没有发生，我喜欢以前的样子”的人有很多同理心。

Well, again, I would say some creative people are very upset. Some creatives are like, "This is the most amazing tool ever, I'm doing incredible new work." But you know like it’s definitely a change. And I have a lot of like empathy to people who are just like, "I wish this change weren't happening. I liked the way things were before."

但原则上，你可以从任何给定的提示中计算出多少……应该有某种方法可以计算出订阅收入或其他收入的多少百分比分配给每个答案。原则上，如果能弄清楚其他规则，这是可能的。显然很复杂。你可以计算某种收入分成，不是吗？

But in principle, you can calculate from any given prompt how much ... there should be some way of being able to calculate what percentage of a subscription, revenue or whatever goes towards each answer. In principle, it should be possible if one could get the rest of the rules figured out. It's obviously complicated. You could calculate some kind of revenue share, no?

如果你是一个音乐家，你一生、整个童年都在听音乐，然后你有了一个想法，创作了一首受之前听过的东西启发但方向新颖的歌曲，你很难说，这部分来自我 11 岁时听到的这首歌。

If you're a musician and you spend your whole life, your whole childhood, listening to music, and then you get an idea and you go compose a song that is inspired by what you've heard before, but a new direction, it'd be very hard for you to say like, this much was from this song I heard when I was 11.

没错。但我们这里讨论的是有人在提示中明确提到某个人的情况。

That's right. But we're talking here about the situation where someone specifically in a prompt names someone.

所以，我认为世界应该帮你快速找到那个模式。而且我认为这会产生巨大的影响。我想快速切换话题。（掌声）你的模型与开源之间的较量。DeepSeek 的出现让你有多动摇？

So, I think the world should help you figure out that model quickly. And I think it will make a huge difference actually. I want to switch topics quickly. (Applause) The battle between your model and open source. How much were you shaken up by the arrival of DeepSeek?

我认为开源有重要的地位。实际上，就在昨晚，我们举办了第一次社区会议，来讨论我们开源模型的参数以及如何塑造它。我们将推出一个非常强大的开源模型。我认为这很重要。我们会做一些接近前沿的东西，比目前任何开源模型都要好。这不会是全部——会有人以在座某些人、也许你或我不喜欢的方式使用它。但开源模型作为这个生态系统的一部分，将会有重要的位置。而且，我觉得我们在这方面行动晚了，但我们现在会做得很好。

I think open source has an important place. We actually, just last night, hosted our first community session to kind of decide the parameters of our open-source model and how we want to shape it. We're going to do a very powerful open-source model. I think this is important. We're going to do something near the frontier, I think, better than any current open-source model out there. This will not be all -- there will be people who will use this in ways that some people in this room, maybe you or I, don’t like. But there is going to be an important place for open-source models as part of the constellation here. And, you know, I think we were late to act on that, but we're going to do it really well now.

我的意思是，你花的钱似乎比 DeepSeek 声称的花费多一个数量级甚至更多，尽管我知道这有争议。你确信更好的模型会得到认可吗？或者说，这难道不是对那种通过大规模投入、数百亿美元投资来维持惊人领先地位的想法构成了某种威胁吗？

I mean, you're spending it seems, like an order, or even orders of magnitude more than DeepSeek allegedly spent, although I know there's controversy around that. Are you confident that the actual better model is going to be recognized? Or are you actually like, isn't this in some ways life-threatening to the notion that, yeah, by going to massive scale, tens of billions of dollars of investment, we can maintain an incredible lead.

我整天都在打电话求人给我们 GPU。我们太受限制了。我们的增长是这样的。DeepSeek 发布了，似乎并没有影响它。还有其他事情在发生。

All day long, I call people and beg them to give us their GPUs. We are so incredibly constrained. Our growth is going like this. DeepSeek launched, and it didn’t seem to impact it. There's other stuff that's happening.

跟我们说说增长情况吧。你在后台告诉了我一个惊人的数字。

Tell us about the growth, actually. You gave me a shocking number backstage there.

我从未在任何公司见过这样的增长，无论是我参与过的还是没有参与过的，就像 ChatGPT 的增长。这真的很有趣。我感到非常荣幸。但经历这一切很疯狂，我们的团队筋疲力尽，压力很大。我们正在努力维持。

I have never seen growth in any company, one that I've been involved with or not, like this, like the growth of ChatGPT. It's really fun. I feel like great, deeply honored. But it is crazy to live through, and our teams are exhausted and stressed. And we’re trying to keep things up.

你们现在有多少用户？

How many users do you have now?

我记得上次我们说的是 5 亿周活跃用户，而且增长非常迅速。

I think the last time we said was 500 million weekly actives, and it is growing very rapidly.

我的意思是，你告诉我这个数字在几周内就翻了一番。是在算力方面还是……

I mean, you told me that like doubled in just a few weeks. Like in terms of compute or in terms of ...

我私下说过，但我想……

I said that privately, but I guess ...

哦。（笑声）我记错了，Sam，对不起。如果你真的想，我们可以把那段剪掉。而且这里没人会发推文。

Oh. (Laughter) I misremembered, Sam, I'm sorry. We can edit that out of the thing if you really want to. And no one here would tweet it.

它增长得非常快。（笑声）

It's growing very fast. (Laughter)

所以你很有信心，你看到它像火箭一样起飞，你一直在发布令人难以置信的新模型。你现在在最好的内部模型中看到了什么还没有与世界分享，但你想在这个舞台上分享的？

So you're confident, you're seeing it grow, take off like a rocket ship, you're releasing incredible new models all the time. What are you seeing in your best internal models right now that you haven't yet shared with the world but you would love to here on this stage?

首先，你问我们是否担心这个或那个模型。世界上会有很多智能模型。非常聪明的模型会在某种程度上商品化。我认为我们会拥有最好的，某些用途你会需要它。但说实话，模型现在非常聪明，对于大多数人想做的大多数事情，它们已经足够好了。我希望随着时间的推移，人们会提高期望，这也会改变。但如果你是一个普通用户使用 ChatGPT，模型能力已经非常聪明了。但我们必须打造一个伟大的产品，而不仅仅是一个伟大的模型。所以会有很多人拥有伟大的模型，而我们会努力打造最好的产品。人们想要图像生成，你之前看到了一些 Sora 视频的例子。他们想把它和所有东西集成。我们刚刚发布了一个新功能，仍然叫 Memory，但比以前好得多，这个模型会在你的一生中逐渐了解你。我们还有很多东西要来打造这样一个伟大的集成产品。而且，我认为人们会坚持使用它。所以会有很多模型，但我希望我们会继续专注于打造这个领域最好的定义性产品。

So first of all, you asked about, are we worried about this model or that model? There will be a lot of intelligent models in the world. Very smart models will be commoditized to some degree. I think we’ll have the best, and for some use you'll want that. But like, honestly, the models are now so smart that for most of the things most people want to do, they're good enough. I hope that'll change over time because people will raise their expectations. But like, if you're kind of using ChatGPT as a standard user, the model capability is very smart. But we have to build a great product, not just a great model. And so there will be a lot of people with great models, and we will try to build the best product. And people want their image-gen, you know, you saw some Sora examples for video earlier. They want to integrate it with all their stuff. We just launched a new feature, it's still called Memory, but it's way better than the Memory before, where this model will get to know you over the course of your lifetime. And we have a lot more stuff to come to build like this great integrated product. And, you know, I think people will stick with that. So there will be many models, but I think we will, I hope, continue to focus on building the best defining product in this space.

我的意思是，昨天我看到你们的公告说 ChatGPT 会知道你所有的查询历史，我输入了“ChatGPT，根据你所知道的一切，告诉我关于我的事。”我惊呆了，Sam，太震撼了。它知道我是谁，以及所有这些兴趣，希望大部分都是合适且可分享的。但太惊人了。我感到真正的兴奋，有点不安，但主要是兴奋，因为它能让我更有用。

I mean after I saw your announcement yesterday that ChatGPT will know all of your query history, I entered, "Tell me about me, ChatGPT, from all you know." And my jaw dropped, Sam, it was shocking. It knew who I was and all these sort of interests that hopefully mostly were pretty much appropriate and shareable. But it was astonishing. And I felt the sense of real excitement, a little bit queasy, but mainly excitement, actually, at how much more that would allow it to be useful to me.

我们的一位研究员在昨天或今天早上发推文说，上传是逐步发生的。不是某一天你把大脑插进去。但你会在一生中与 ChatGPT 交谈，有一天，也许如果你愿意，它会整天听你说话，观察你在做什么，它会了解你，成为你自身的延伸，成为你的伴侣，一个试图帮助你成为最好、尽你所能的东西。

One of our researchers tweeted, you know, kind of like yesterday or this morning, that the upload happens bit by bit. It’s not you know, that you plug your brain in one day. But you will talk to ChatGPT over the course of your life and some day, maybe if you want, it'll be listening to you throughout the day and sort of observing what you're doing, and it'll get to know you and it'll become this extension of yourself, this companion, this thing that just tries to, like, help you be the best, do the best you can.

在电影《她》中，AI 基本上宣布她读了他所有的邮件，并认定他是个伟大的作家，然后说服出版商出版他的作品。这可能比我们想象的来得更快。

In the movie "Her," the AI basically announces that she's read all of his emails and decided he's a great writer and you know, persuades a publisher to publish him. That might be coming sooner than we think.

我不认为它会完全那样发生，但是的，我认为朝着 AI 的方向——你不需要只是去 ChatGPT 或别的什么说，我有一个问题，给我答案。而是你会主动收到推送，帮助你、让你变得更好的东西。这似乎很快就会到来。

I don't think it will happen exactly like that, but yeah, I think something in the direction where AI -- you don’t have to just, like, go to ChatGPT or whatever and say, I have a question, give me an answer. But you're getting like, proactively pushed things that help you, that make you better or whatever. That does seem like it's soon.

那么你在内部看到了什么即将到来的东西，会让你觉得会让人大吃一惊？至少给我们一点提示，下一个让人瞠目结舌的东西是什么？

So what have you seen that's coming up, internally, that you think is going to blow people's minds? Give us at least a hint of what the next big jaw dropper is.

我个人目前最兴奋的是 AI 用于科学。我坚信世界和人们生活越来越好的最重要驱动力是新的科学发现。我们可以用更少的资源做更多的事情，我们推动可能性的边界。我们开始听到很多科学家说，使用我们的最新模型，他们实际上比以前更高效了。这确实对他们能发现什么很重要。

The thing that I'm personally most excited about is AI for science at this point. I am a big believer that the most important driver of the world and people's lives getting better and better is new scientific discovery. We can do more things with less, we sort of push back the frontier of what's possible. We're starting to hear a lot from scientists with our latest models that they're actually just more productive than they were before. That's actually mattering to what they can discover.

什么可能是近期的发现，比如室温——

What’s the plausible near-term discovery, like, room temperature --

超导体？

Superconductors?

超导，是的。有可能吗？

Superconducting, yeah. Is that possible?

我不认为这被物理定律所禁止。所以应该是可能的。但我们不确定。我认为你会开始看到一些……借助 AI 辅助工具在对抗疾病方面取得有意义的进展。

I don't think that's prevented by the laws of physics. So it should be possible. But we don't know for sure. I think you'll start to see some ... meaningful progress against disease with AI-assisted tools.

你知道，物理学可能需要更长一点时间，但我对此抱有希望。所以这是一个方向。另一个我认为很重要的方向很快就会开始，就在未来几个月。软件开发已经发生了很大的变化。现在创建软件的过程与两年前相比，真是令人惊叹。但我预计在未来几个月，随着智能体式软件工程真正开始，还会有同样大的变化。

You know, physics maybe takes a little bit longer, but I hope for it. So that's like, one direction. Another that I think is big is starting pretty soon, like in the coming months. Software development has already been pretty transformed. Like it’s quite amazing how different the process of creating software is now than it was two years ago. But I expect like another move that big in the coming months as agentic software engineering really starts to happen.

我听到工程师们说，他们使用一些新模型时几乎有了宗教般的体验，突然之间，他们一个下午就能完成过去需要两年才能完成的工作。

I've heard engineers say that they've had almost like religious-like moments with some of the new models where suddenly, they can do in an afternoon what would have taken them two years.

是的，这真是令人难以置信——这确实是我感受到 AGI 的重大时刻之一。

Yeah, it's like mind -- it really like, that’s been one of my big “feel the AGI” moments.

但谈谈你见过的最可怕的事情。因为外界很多人把你描绘成，你知道，你能接触到这些东西。我们听到各种关于 AI 的谣言，比如“天哪，他们看到了意识”，或者“他们看到了 AGI”，或者“他们看到了某种末日来临”。你有没有见过，有没有某个可怕的时刻，你在内部看到某些东西，然后想，“哦，我们需要关注这个？”

But talk about what is the scariest thing that you've seen. Because like, outside, a lot of people picture you as, you know, you have access to this stuff. And we hear all these rumors coming out of AI, and it's like, "Oh my God, they've seen consciousness," or "They've seen AGI," or "They've seen some kind of apocalypse coming." Have you seen, has there been a scary moment when you've seen something internally and thought, "Uh oh, we need to pay attention to this?"

有过一些令人敬畏的时刻。随之而来的总是，这能走多远？这会变成什么？但并没有，我们没有秘密地拥有一个具有意识的模型，或者能够自我改进的东西。你知道，我……人们对 AI 的重大风险有不同的看法。我自己对风险出现在哪里的想法也在演变。我仍然相信会出现非常强大的模型，人们可能会以大规模的方式滥用它们。人们经常谈论新型生物恐怖主义的可能性，能够带来真正网络安全挑战的模型，以及能够以某种方式自我改进导致失控的模型。所以我认为那里存在重大风险。还有很多其他事情，老实说，我认为很多人指的是虚假信息或模型说出他们不喜欢的话之类的事情。

There have been like moments of awe. And I think with that is always like, how far is this going to go? What is this going to be? But there's no like, we don't secretly have, we're not secretly sitting on a conscious model or something that's capable of self-improvement or anything like that. You know, I ... people have very different views of what the big AI risks are going to be. And I myself have like evolved on thinking about where we're going to see those. I continue to believe there will come very powerful models that people can misuse in big ways. People talk a lot about the potential for new kinds of bioterror, models that can present like a real cybersecurity challenge, models that are capable of self-improvement in a way that leads to some sort of loss of control. So I think there are big risks there. And then there's a lot of other stuff, which honestly is kind of what I think, many people mean, where people talk about disinformation or models saying things that they don't like or things like that.

先说第一个，你们在发布前会在内部检查吗？

Sticking with the first of those, do you check for that internally before release?

当然，是的。我们有一个预备框架，概述了我们如何做到这一点。

Of course, yeah. So we have this preparedness framework that outlines how we do that.

我的意思是，你的安全团队有一些人离职。有多少人离开了，他们为什么离开？

I mean, you've had some departures from your safety team. How many people have departed, why have they left?

我们有一些，我不知道确切数字，但显然对 AI 安全系统有不同的看法。我真的想指出我们的记录。有人会说各种话。你知道，现在世界上大约有 10%的人大量使用我们的系统。我们为安全记录感到非常自豪。

We have, I don't know the exact number, but there are clearly different views about AI safety systems. I would really point to our track record. There are people who will say all sorts of things. You know, something like 10 percent of the world uses our systems now a lot. And we are very proud of the safety track record.

但从某种意义上说，记录并不是问题——

But track record isn't the issue in a way --

不，它确实是。

No, it kind of is.

因为我们谈论的是指数级增长的力量，我们担心有一天醒来世界末日就到了。所以这真的不是关于记录，而是关于合理地说明，如果我们看到危险，我们有能力迅速关闭系统。

Because we're talking about an exponentially growing power where we fear that we may wake up one day and the world is ending. So it's really not about track record, it's about plausibly saying that the pieces are in place to shut things down quickly if we see a danger.

是的，不，当然，当然这很重要。你不会某天醒来就说：“嘿，我们没有任何安全流程。现在我们认为模型非常聪明。所以我们现在必须关心安全。”你必须在这条指数曲线上一直关心安全。当然风险在增加，存在巨大挑战。但我们学习如何构建安全系统的方式是一个迭代过程：将它们部署到世界，获得反馈，在风险相对较低的时候，了解哪些是我们必须解决的问题。我认为随着我们进入这些智能体系统，有一大类新事物我们必须学会应对。

Yeah, no, of course, of course that's important. You don't, like, wake up one day and say, "Hey, we didn't have any safety process in place. Now we think the model is really smart. So now we have to care about safety." You have to care about it all along this exponential curve. Of course the stakes increase, and there are big challenges. But the way we learn how to build safe systems is this iterative process of deploying them to the world, getting feedback, while the stakes are relatively low, learning about like, this is something we have to address. And I think as we move into these agentic systems, there's a whole big category of new things we have to learn to address.

那么让我们谈谈智能体系统以及它与 AGI 的关系。我认为存在困惑，我自己也很困惑。所以人工通用智能，感觉 ChatGPT 已经是一种通用智能了。我可以问它任何问题，它都会给出智能的回答。为什么那不是 AGI？

So let's talk about agentic systems and the relationship between that and AGI. I think there's confusion out there, I'm confused. So artificial general intelligence, it feels like ChatGPT is already a general intelligence. I can ask it about anything, and it comes back with an intelligent answer. Why isn't that AGI?

它并不……首先，你不能问它任何问题。你这么说很好，但它在很多事情上仍然糟糕得令人尴尬。但即使我们修复了这些问题，希望我们会做到，它也不能持续学习和改进。它不能在自己目前薄弱的方面变得更好。它不能去发现新的科学并更新自己的理解。而且它也不能，即使我们降低标准，它也不能像你在电脑前那样完成任何知识工作。实际上，即使没有那种在未知领域自我改进的能力，我可能会接受那作为 AGI 的定义。但目前的系统，你不能说，嘿，去帮我做这个工作任务，然后它就去网上点击、打电话、查看你的文件并完成它。没有这些，它显然还不够格。

It doesn't ... First of all, you can't ask it anything. That's very nice of you to say, but there's a lot of things that it's still embarrassingly bad at. But even if we fixed those, which hopefully we will, it doesn't continuously learn and improve. It can't go get better at something that it's currently weak at. It can't go discover new science and update its understanding and do that. And it also kind of can't, even if we lower the bar, it can't just sort of do any knowledge work you could do in front of a computer. I actually, even without the sort of ability to get better at something it doesn't know yet, I might accept that as a definition of AGI. But the current systems, you can't say like, hey, go do this task for my job, and it goes off and clicks around the internet and calls someone and looks at your files and does it. And without that, it feels definitely short of it.

我的意思是，你们内部有没有一个明确的 AGI 定义，你认为我们什么时候能达到？

I mean, do you guys have internally a clear definition of what AGI is, and when do you think that we may be there?

就像那个笑话，如果你把 10 个 OpenAI 研究人员放在一个房间里，让他们定义 AGI，你会得到 14 个定义。

It's like the joke, if you’ve got 10 OpenAI researchers in a room and ask to define AGI, you’d get 14 definitions.

但这很令人担忧，不是吗？因为那最初就是使命，“我们将成为第一个实现 AGI 的。我们会安全地做到。但我们没有一个明确的定义。”

That's worrying, though, isn't it? Because that has been the mission initially, “We’re going to be the first to get to AGI. We'll do so safely. But we don't have a clear definition of what it is."

我本来要说完答案的。

I was going to finish the answer.

抱歉。

Sorry.

我认为重要的是，人们想知道的不在于“我们完成了”这个神奇时刻在哪里。但鉴于看起来会发生的是，模型会变得越来越聪明、越来越有能力，在这条漫长的指数曲线上，不同的人会在不同时间点称之为 AGI。但我们都同意它会远远超越那个点。你知道，无论你如何称呼这些比我们更有能力的系统。重要的是我们如何谈论一个系统，它在所有这些步骤及之后都是安全的，当系统变得比我们更有能力，当系统能做我们不完全理解的事情时。我认为比 AGI 何时到来以及它的定义更重要的是，认识到我们正处于这条不可思议的指数曲线上。你可以说这是我认为的 AGI。你可以说你认为这是你心中的 AGI。其他人可以说超级智能就在这里，但我们将不得不应对并从这一不可思议的系统中获得巨大收益。所以我认为我们应该把对话从“AGI 时刻”转向认识到，这个东西不会停止，它会远远超越我们任何人称之为 AGI 的东西。

What I think matters though, and what people want to know is not where is this one, you know, magic moment of, “We finished.” But given that what looks like is going to happen is that the models are just going to get smarter and more capable and smarter and more capable, on this long exponential, different people will call it AGI at different points. But we all agree it’s going to go way, way past that. You know, to whatever you want to call these systems that get much more capable than we are. The thing that matters is how do we talk about a system that is safe through all of these steps and beyond, as the system gets more capable than we are, as the system can do things that we don't totally understand. And I think more important than when is AGI coming and what's the definition of it, it's recognizing that we are in this unbelievable exponential curve. And you can, you know, say this is what I think AGI is. You can say you think this is what you think AGI is. Someone else can say superintelligence is out here, but we're going to have to contend and get wonderful benefits from this incredible system. And so I think we should shift the conversation away from what's the AGI moment to a recognition that, like, this thing is not going to stop, it's going to go way beyond what any of us would call AGI.

我们必须建立一个社会来获得这项技术的巨大好处，并想办法确保其安全。嗯，这周的一个讨论是，真正的变革时刻是——我的意思是，AGI 是个模糊的概念，但明确的是智能体式 AI——当 AI 被放开去自主执行任务并整合资源时——实际上，你们有一个叫 Operator 的东西已经开始这样做了。我试了一下。你知道，我想订个餐厅，它挺神奇的。它基本上能帮你完成，但它是这么说的。这是一个有趣的过程。然后它说“给我你的信用卡”之类的，我这次拒绝了继续。但我认为这就是人们将要面临的挑战。这有点像，它是一种不可思议的超能力，也有点吓人。Yoshua Bengio 在这里发言时说，智能体式 AI 是值得关注的事情。当我们赋予 AI 权力去互联网上做事时，一切可能出错。我的意思是，在科幻故事中，进入互联网总是那个时刻，你知道，逃脱发生，事情可能变得非常糟糕。你如何既发布智能体式 AI，又设置护栏防止它越界？

And we have to build a society to get the tremendous benefits of this and figure out how to make it safe. Well, one of the conversations this week has been that the real change moment is -- I mean, AGI is a fuzzy thing, but what is clear is agentic AI -- when AI is set free to pursue projects on its own and to put the pieces together -- you’ve actually, you've got a thing called Operator which starts to do this. And I tried it out. You know, I wanted to book a restaurant, and it's kind of incredible. It kind of can go ahead and do it, but this is what it said. You know, it was an intriguing process. And, you know, “Give me your credit card” and everything else, and I declined on this case to go forward. But I think this is the challenge that people are going to have. It's kind of like, it's an incredible superpower. It's a little bit scary. And Yoshua Bengio, when he spoke here, said that agentic AI is the thing to pay attention to. This is when everything could go wrong as we give power to AI to go out onto the internet to do stuff. I mean, going out onto the internet was always, in the sci-fi stories, the moment where, you know, escape happened and potential -- things could go horribly wrong. How do you both release agentic AI and have guardrails in place that it doesn't go too far?

首先，显然你可以选择不这样做，说我不想要这个。我会打电话给餐厅，在电话里念信用卡号。

First of all, obviously you can choose not to do this and say, I don't want this. I'm going to call the restaurant and read them my credit card over the phone.

我可以选择，但别人可能会说，“哦，ChatGPT，去互联网上把整个网络改写得对人类更好”之类的。

I could choose, but someone else might say, “Oh, go out, ChatGPT onto the internet at large and rewrite the internet to make it better for humans,” or whatever.

我想说的是，任何新技术都需要一段时间才能让人们适应。我记得以前我不愿意在网上输入信用卡号，因为我父母让我相信会有人偷看号码，你必须填好表格然后打电话给他们。后来我们都觉得，好吧，我们会建立反欺诈系统，然后就能放心了。我认为人们在很多方面都会慢慢适应智能体式 AI。但我也非常同意你说的，即使有些人适应而有些人不适应，我们仍然会有 AI 系统在互联网上到处点击。我认为这是我们面临的最有趣、也最重要的安全挑战。因为当你让 AI 访问你的系统、你的信息、在电脑上点击的能力时，当 AI 犯错时，风险要高得多。这就像一道门槛——我们之前谈过安全和能力。我越来越觉得它们变成了一维的。好的产品就是安全的产品。如果你不信任我们的智能体不会清空你的银行账户、删除你的数据或做其他坏事，你就不会使用它们。所以人们想使用真正可信、真正安全的智能体。我认为我们的进展受限于我们做到这一点的能力。但这是产品的基本组成部分。

The point I was going to make is just with any new technology, it takes a while for people to get comfortable. I remember when I wouldn't put my credit card on the internet because my parents had convinced me someone was going to read the number, and you had to fill out the form and then call them. And then we kind of all said, OK, we’ll build anti-fraud systems, and we can get comfortable with this. I think people are going to be slow to get comfortable with agentic AI in many ways. But I also really agree with what you said, which is that even if some people are comfortable with it and some aren't, we are going to have AI systems clicking around the internet. And this is, I think, the most interesting and consequential safety challenge we have yet faced. Because AI that you give access to your systems, your information, your ability to click around on your computer, now, those, you know, when AI makes a mistake, it's much higher stakes. It is the gate on -- so we talked earlier about safety and capability. I kind of think they're increasingly becoming one-dimensional. Like a good product is a safe product. You will not use our agents if you do not trust that they’re not going to like empty your bank account or delete your data or who knows what else. And so people want to use agents that they can really trust, that are really safe. And I think we are gated on our ability to make progress on our ability to do that. But it's a fundamental part of the product.

在一个智能体无处不在的世界里，比如说，也许开放模型被广泛分发，有人说，“好吧，AGI，我要你去互联网上，用任何方式传播一个梗，说 X 人群是邪恶的”之类的。这不一定是个人选择。一个人就可以放出那个智能体，然后智能体可能决定，“为了执行这个功能，我得把自己复制到各处”。你们内部有没有明确划定的红线，知道哪些是危险时刻，不能发布可能越界的东西？

In a world where agency is out there and say that, you know, maybe it’s open models are widely distributed and someone says, "OK, AGI, I want you to go out onto the internet and, you know, spread a meme however you can that X people are evil," or whatever it is. It doesn't have to be an individual choice. A single person could let that agent out there, and the agent could decide, "Well, in order to execute on that function, I've got to copy myself everywhere," and, you know. Are there red lines that you have clearly drawn internally, where you know what the danger moments are, and that we cannot put out something that could go beyond this?

是的，这就是我们准备框架的目的。我们会随时间更新。但我们试图勾勒出我们认为最重要的危险时刻，或者类别是什么，如何衡量，以及如何在发布前缓解风险。从对话中我能看出你不是一个 AI 的狂热粉丝。

Yeah, so this is the purpose of our preparedness framework. And we'll update that over time. But we’ve tried to outline where we think the most important danger moments are, or what the categories are, how we measure that, and how we would mitigate something before releasing it. I can tell from the conversation you're not a big AI fan.

实际上恰恰相反，我每天都用它。我对它充满敬畏。我认为这是一个不可思议的时代。我不想活在别的时代，我迫不及待想看到它的发展。我们一直持有……我认为持有这种态度至关重要……我们不能把人分成两个阵营。你必须对可能性抱有热情的信念，但不要被它过度迷惑，因为事情可能变得非常糟糕。（掌声）

Actually, on the contrary, I use it every day. I'm awed by it. I think this is an incredible time to be alive. I wouldn't be alive any other time, and I cannot wait to see where it goes. We've been holding ... I think it's essential to hold ... like we can’t divide people into those camps. You have to hold a passionate belief in the possibility, but not be overseduced by it because things could go horribly wrong. (Applause)

我想说的是，我完全理解。我完全理解看着这一切，说这是世界即将发生的难以置信的变化。也许我不想要这个，或者我喜欢其中一部分。也许我喜欢和 ChatGPT 聊天，但我担心艺术会变成什么样，担心变化的速度，担心这些智能体在互联网上到处点击。也许总的来说，我希望这一切没有发生。或者我希望它发生得慢一点。或者我希望它以一种我能挑选哪些进步发生的方式发生。我认为恐惧是完全理性的。焦虑是完全理性的。我们自己也有很多这样的情绪。但是……第一，会有巨大的好处。显然，你每天都用它，你喜欢它。第二……我真的相信，随着时间的推移，社会会在犯一些大错误的过程中，学会如何正确对待技术。第三，这将会发生。这就像发现了一个基础物理定律，现在全世界都知道了。它将成为我们世界的一部分。我认为这个对话非常重要。我认为讨论这些危险领域非常重要。新的经济模式非常重要。但我们必须以谨慎而非恐惧的态度来拥抱它，否则我们会被那些用 AI 做得更好的人甩在后面。

What I was going to say is I totally understand that. I totally understand looking at this and saying this is an unbelievable change coming to the world. And, you know, maybe I don't want this, or maybe I love parts of it. Maybe I love talking to ChatGPT, but I worry about what's going to happen to art, and I worry about the pace of change, and I worry about these agents clicking around the internet. And maybe, on balance, I wish this weren't happening. Or maybe I wish it were happening a little slower. Or maybe I wish it were happening in a way where I could pick and choose what parts of progress were going to happen. And I think, the fear is totally rational. Sort of, the anxiety is totally rational. We all have a lot of it, too. But ... A, there will be tremendous upside. Obviously, you know, you use it every day, you like it. B ... I really believe that society figures out, over time, with some big mistakes along the way, how to get technology right. And C, this is going to happen. This is like a discovery of fundamental physics that the world now knows about. And it's going to be part of our world. And I think this conversation is really important. I think talking about these areas of danger really important to talk about. New economic models are really important. But we have to embrace this with like, caution but not fear, or we will get run by with other people that use AI to do better.

你实际上是安全方面最有说服力的倡导者之一。你在参议院作证过。我记得你基本上说我们应该成立一个新的安全机构，对所有努力进行许可，也就是说它会拒绝许可某些努力。你还相信这个政策提案吗？

You've actually been one of the most eloquent proponents of safety. You testified in the Senate. I think you said basically that we should form a new safety agency that licenses any effort, ie. it will refuse to license certain efforts. Do you still believe in that policy proposal?

我对政府运作有了更多了解。我不认为这是完全正确的政策提案。

I have learned more about how the government works. I don't think this is quite the right policy proposal.

那正确的政策提案是什么？

What is the right policy proposal?

但我确实认为，随着这些系统越来越先进并产生真正的全球影响，我们需要某种方式，也许公司自己制定合适的框架或模型，但我们需要某种方式让非常先进的模型接受外部安全测试。并且我们了解何时接近这些危险区域。我仍然非常相信这一点。

But, I do think the idea that as these systems get more advanced and have legitimate global impact, we need some way, you know, maybe the companies themselves put together the right framework or the right sort of model for this, but we need some way that very advanced models have external safety testing. And we understand when we get close to some of these danger zones. I very much still believe in that.

我觉得讽刺的是，我们想要的可能是安全机构，而智能体恰恰是不安全的东西。

Struck me as ironic that a safety agency might be what we want, and yet agency is the very thing that is unsafe.

那么 Sam，我问了你的 o1-pro 推理模型，它非常——

So Sam, I asked your o1-pro reasoning model, which is incredibly --

谢谢你的 200 美元。

Thank you for the 200 dollars.

（笑）每月 200 美元。这价格很划算。我问它，我能问你的最尖锐的问题是什么？它思考了两分钟。两分钟。你想看这个问题吗？

(Laughs) 200 dollars a month. It's a bargain at the price. I said, what is the single most penetrating question I could ask you? It thought about it for two minutes. Two minutes. You want to see the question?

想。

I do.

“Sam，既然你在帮助创造可能重塑整个人类命运的技术，是谁赋予了你（或任何人）这样做的道德权威？”（笑声）“如果你错了，你个人如何负责？”

"Sam, given that you're helping create technology that could reshape the destiny of our entire species, who granted you (or anyone) the moral authority to do that?" (Laughter) "And how are you personally accountable if you're wrong?"

不，这个问题很好。

No, it was good.

那很令人印象深刻。

That was impressive.

你过去半小时一直在问我类似的问题。你觉得呢？（笑声和掌声）

You've been asking me versions of this for the last half hour. What do you think? (Laughter and applause)

我会这样说。这是我版本的那个问题。

What I would say is this. Here's my version of that question.

但没有答案？

But no answer?

你问我的问题是什么？

What was your question to me?

你会怎么回答那个问题？

How would you answer that one?

站在你的角度？

In your shoes?

或者作为旁观者。

Or as an outsider.

我不知道。我对你感到困惑。我有点敬畏你。因为你建造了最令人惊叹的东西之一。关于你有两种说法。一种是，你是一个不可思议的远见者，完成了不可能的事，震惊了世界。用比 Google 少得多的人，你推出了比任何现有东西都强大得多的产品。我的意思是，你建造的东西太惊人了。但另一种说法是，你改变了立场，从 OpenAI 这个开放的东西，转向了建造超级强大东西的诱惑。而且，你失去了一些关键人物。外面有这种说法。有些人认为你在 AI 领域不可信任。我很想知道你是谁。你对自己的叙述是什么？你的核心价值观是什么，Sam，能让我们、让世界相信，拥有如此大权力的人配得上它？

I don't know. I am puzzled by you. I’m kind of awed by you. Because you’ve built one of the most astonishing things out there. There are two narratives about you out there. One is, you know, you are this incredible visionary who's done the impossible, and you shocked the world. With far fewer people than Google, you came out with something that was much more powerful than anything being done. I mean, it is amazing what you've built. But the other narrative is that you have shifted ground, that you've shifted from being OpenAI, this open thing, to the allure of building something super powerful. And you know, you’ve lost some of your key people. There's a narrative out there. Some people believe that you're not to be trusted in this space. I would love to know who you are. What is your narrative about yourself? What are your core values, Sam, that can give us, the world, confidence that someone with so much power here is entitled to it?

听着，我认为像其他人一样，我是一个复杂的角色，不能简单地归结为一个维度。你知道，可能有些好的方面是真的，有些批评也是真的。就 OpenAI 而言，我们的目标是制造 AGI 并分发它，确保安全，造福全人类。我认为从各方面来看，我们在那个方向上做了很多。显然我们的策略随着时间的推移发生了变化。我认为我们当时并不真正知道自己长大后要成为什么。我们没想到必须围绕这个建立一家公司。我们学到了很多关于事情如何发展以及这些系统需要多少资本的现实。但就把极其强大的 AI 以高安全性交到很多人手中，并给他们工具去做各种了不起的事情而言，我认为很难给我们打低分。我确实认为我们应该更多地开源，这是合理的。我认为出于你之前问到的所有原因，当我们不确定这些系统的影响以及如何确保安全时，我们采取谨慎行动是合理的。我认为你之前的很多问题至少表明你对我们那样运作有一定理解。但现在我认为世界有了更好的理解，是时候把非常强大的开放系统推向世界了。如果你明年再邀请我，你可能会因为有人滥用这些开源系统而对我大喊大叫，说“你为什么那样做？”你知道，“你不应该回到你的开放根源。”但是，你知道，我们所做的一切都有权衡。我们是 AI 革命中的一个参与者、一个声音，尽力做到最好，并以负责任的方式将这项技术引入世界。我们肯定犯过错误，未来还会犯更多错误。总的来说，我认为在过去近十年里，我们已经基本完成了我们设定要做的事情。我们前面还有很长的路要走。我们的策略未来还会改变，但对使命和目标的坚持，我认为非常坚定。（掌声）

Look, I think like anyone else, I'm a nuanced character that doesn't reduce well to one dimension here. You know, probably some of the good things are true and probably some of the criticism is true. In terms of OpenAI, our goal is to make AGI and distribute it, make it safe, for the broad benefit of humanity. I think by all accounts, we have done a lot in that direction. Clearly our tactics have shifted over time. I think we didn't really know what we were going to be when we grew up. We didn't think we would have to build a company around this. We learned a lot about how it goes and the realities of what these systems were going to take from capital. But I think we've been, in terms of putting incredibly capable AI with a high degree of safety in the hands of a lot of people, and giving them tools to sort of do whatever amazing things they're going to do, I think it'd be hard to give us a bad grade on that. I do think it's fair that we should be open sourcing more. I think it was reasonable for all of the reasons that you asked earlier, as we weren't sure about the impact these systems were going to have and how to make them safe, that we acted with precaution. I think a lot of your questions earlier would suggest at least some sympathy to the fact that we've operated that way. But now I think we have a better understanding, as a world, and it is time for us to put very capable open systems out into the world. If you invite me back next year, you will probably yell at me for somebody who has misused these open-source systems, and say, "Why did you do that?" You know, "You should have not gone back to your open roots." But, you know, there's trade offs in everything we do. And we are one player, one voice, in this AI revolution, trying to do the best we can and kind of steward this technology into the world in a responsible way. We've definitely made mistakes, we'll definitely make more in the future. On the whole, I think we have, over the last almost decade, it’s been a long time now, you know, we have mostly done the thing we’ve set out to do. We have a long way to go in front of us. Our tactics will shift more in the future, but adherence to this sort of mission and what we're trying to do, I think very strong. (Applause)

你发过这个——好吧，这是《指环王》中的权力之戒。你的对手，我会说，不是你目前最好的朋友，埃隆·马斯克声称，他认为你被权力之戒腐蚀了。顺便说一句，这个指控——（笑声）

You posted this -- Well, OK, so here's the Ring of Power from "Lord of the Rings." Your rival, I will say, not your best friend at the moment, Elon Musk, claimed that, you know, he thought that you'd been corrupted by the Ring of Power. An allegation that, by the way -- (Laughter)

嗨，Steve。

Hi, Steve.

一个同样可以适用于埃隆的指控，你知道，公平地说。但我很好奇，人们，你——

An allegation that could be applied to Elon as well, you know, to be fair. But I'm curious, people, you have --

我可能会回应。我在考虑。我可能会说点什么。（笑声）

I might respond. I'm thinking about it. I might say something. (Laughter)

这是每个人都在想的，当我们看到科技 CEO 变得更有权势、更富有时，他们能处理好，还是变得不可抗拒？权力和财富是否有时让人无法做正确的事，而你必须紧紧抓住那枚戒指？你怎么看？我是说，你有时感觉到那枚戒指吗？

It's in everyone's mind, as we see technology CEOs get more powerful, get richer, is can they handle it, or does it become irresistible? Does the power and the wealth make it impossible to sometimes do the right thing and you just have to cling tightly to that ring? What do you think? I mean, do you feel that ring sometimes?

与其他获得大量权力并改变行为或在世界上做了很多事的 CEO 相比，你觉得我做得怎么样？你怎么看？（掌声）

How do you think I'm doing, relative to other CEOs, that have gotten a lot of power and changed how they act or done a bunch of stuff in the world, like how do you think? (Applause)

你有一个美丽的……你不是一个粗鲁、愤怒、出来对别人说攻击性话的人。

You have a beautiful ... you are not a rude, angry person who comes out and says aggressive things to other people.

有时我会那样。那是我唯一的缺点，你知道？（笑声）

Sometimes I do that. That's my single vice, you know? (Laughter)

不，我认为你个人行事的方式令人印象深刻。我的意思是，有些人问的问题是，那是真实的你，还是有什么别的东西在发生？

No, I think in the way that you personally conduct yourself, it's impressive. I mean, the question some people ask is, is that the real you or, you know, is there something else going on?

不，我接受反馈。你放了索伦的至尊戒或随便什么东西。所以我接受反馈。我做过什么让你觉得我被权力腐蚀了？

No, I'll take the feedback. You put up the Sauron Ring of Power or whatever that thing is. So I'll take the feedback. What is like something I have done where you think I've been corrupted by power?

我认为担忧在于 OpenAI 向盈利模式的转变，你知道，有些人说，看吧。你被财富欲望腐蚀了。你知道，曾经它没有股权。它会让你变得极其富有。顺便说一句，我个人不认为那是你的动机。我认为你想建造非常酷的东西。我担心的是竞争感，你看到别人在做，这让你无法以正确的速度发展。但你告诉我，如果你没有那种感觉，像是什么……世界上很少有人拥有你这样的能力和潜力，我们不知道那是什么感觉。

I think the fear is that just the transition of OpenAI to a for-profit model, is, you know, some people say, well, there you go. You got corrupted by the desire for wealth. You know, at one point there was going to be no equity in it. It will make you fabulously wealthy. By the way, I don't think that is your motivation, personally. I think you want to build stuff that is insanely cool. And what I worry about is the competitive feeling that you see other people doing it, and it makes it impossible to develop at the right pace. But you tell me, if you don't feel that, like, what ... so few people in the world have the kind of capability and potential you have, we don't know what it feels like.

感觉如何？

What does it feel like?

令人惊讶的是，和以前一样。我觉得人可以逐步适应任何事情。如果我从十年前一下子被传送到现在，那会非常迷失。但任何事情都会变成新常态，所以感觉没什么不同。坐在这里谈论这个很奇怪，但日常生活的单调——我是在最好的意义上说的——感觉完全一样。

Shockingly, the same as before. I think you can get used to anything step by step. I think if I were transported from 10 years ago to right now, all at once, it would feel very disorienting. But anything does become sort of the new normal, so it doesn't feel any different. And it's strange to be sitting here talking about this, but the monotony of day-to-day life, which I mean in the best possible way, feels exactly the same.

你还是同一个人。

You're the same person.

我的意思是，我肯定在很多方面不是了，但我感觉没什么不同。

I mean, I'm sure I'm not in all sorts of ways, but I don't feel any different.

你发的关于你儿子的内容很美。你最后说的那句‘我从未感受过这样的爱’，我想在场的任何父母都知道那种感觉，那种人类拥有而 AI 永远不会有的狂野生物本能，当你抱着孩子的时候。我想知道这是否改变了你对事情的看法，比如，这里有一个红盒子，这里有一个带红色按钮的黑盒子，你可以按下那个按钮，给你儿子一个最不可思议的人生，但同时也注入 10%的毁灭风险。你会按吗？

This was a beautiful thing you posted, your son. I mean, that last thing you said there, 'I've never felt love like this,' I think any parent in the room so knows that feeling, that wild biological feeling that humans have and AIs never will, of you’re holding your kid. And I'm wondering whether that's changed how you think about things like if, you know, say, here's a red box, here's a black box with the red button on it, you can press that button and you give your son likely the most unbelievable life, but also you inject a 10 percent chance that he gets destroyed. Do you press that button?

在字面情况下，不会。如果问题是，我是否觉得我在工作中也在做类似的事，答案是，我也不觉得。有孩子改变了很多事情。这是迄今为止发生在我身上最神奇的事，就像所有人说的那样。我的联合创始人 Ilya 曾说过一句话，我记不太清，大意是‘我不知道生命的意义是什么，但肯定和婴儿有关’。这准确得令人难以置信。它改变了我愿意在某些事情上花多少时间，不和儿子在一起的代价高得离谱。但我以前就很关心不毁灭世界，现在也很关心。我不需要孩子来让我关心那部分。我确实更多地思考未来对他会是什么样，但我感到有责任为所有人的未来做最好的事。

In the literal case, no. If the question is, do I feel like I'm doing that with my work, the answer is, I also don't feel like that. Having a kid changed a lot of things. And by far the most amazing thing that has ever happened to me, like everything everybody says is true. A thing my cofounder Ilya said once is, I don’t know. This is a paraphrase, something like, 'I don't know what the meaning of life is, but for sure it has something to do with babies.' And it's like, unbelievably accurate. It changed how much I’m willing to spend time on certain things and the cost of not being with my kid is just crazily high. But I really cared about not destroying the world before. I really care about it now. I didn't need a kid for that part. I definitely think more about what the future will be like for him in particular, but I feel a responsibility to do the best thing I can for the future of everybody.

Tristan Harris 本周在这里做了一个非常有影响力的演讲，他说关键问题在于，你和你的同行们基本上都觉得，高级 AI 的发展是不可避免的，竞赛已经开始，除了尽力赢得比赛并尽可能负责任地去做，别无选择。也许有一种情景是你们的超级智能 AI 可以充当其他人 AI 的刹车。但正是每个人都认为它不可避免这一事实，导致了严重的风险和不稳定。你认为你和你的同行们是否觉得这是不可避免的？你能看到任何出路，让我们可以集体同意放慢一点速度，让整个社会参与进来，说‘不，我们不想让这一切发生得这么快，这太 disruptive 了’吗？

Tristan Harris gave a very powerful talk here this week in which he said that the key problem, in his view, was that you and your peers in these other models all feel basically, that the development of advanced AI is inevitable, that the race is on, and that there is no choice but to try and win that race and to do so as responsibly as you can. And maybe there’s a scenario where your superintelligent AI can act as a brake on everyone else's or something like that. But that the very fact that everyone believes it is inevitable, that is a pathway to serious risk and instability. Do you think that you and your peers do feel that it's inevitable, and can you see any pathway out of that where we could collectively agree to just slow things down a bit, have society as a whole weigh in a bit and say, no, you know, we don't want this to happen quite as fast. It's too disruptive.

首先，我认为人们一直在放慢速度，因为技术还没准备好，因为某些东西不够安全，因为某些东西不奏效。我认为所有努力都在暂停、延迟、不发布某些能力。所以我认为这确实在发生。而且，我认为过往记录很重要。如果我们仓促推出产品，出现各种问题，比如产品不如人意，或者有真正的安全问题等。我会回到我们做的一个改变，我认为你可以做到。大多数努力之间都有沟通，只有一个例外。我认为所有努力都非常关心 AI 安全。而且我认为——

First of all, I think people slow things down all the time because the technology is not ready, because something's not safe enough, because something doesn't work. There are, I think, all of the efforts hold on things, pause on things, delay on things, don't release certain capabilities. So I think this happens. And again, this is where I think the track record does matter. If we were rushing things out and there were all sorts of problems, either the product didn’t work as people wanted it to or there were real safety issues or other things there. And I will come back to a change we made, I think you could do that. There is communication between most of the efforts, with one exception. I think all of the efforts care a lot about AI safety. And I think that --

谁是例外？

Who's the exception?

我不说。我认为大家真的非常关心如何做好这件事。把这件事描绘成疯狂的竞赛或冲刺，忽略了人们正在快速推出模型并为人们制作优秀产品的细微差别。但人们感受到的影响如此之大……如果你能参加 OpenAI 或其他公司的会议，你会觉得‘哦，这些人真的很关心这个’。我们最近确实改变了对传统上被认为是安全的一部分的看法。在我们的新图像模型中，我们给了用户更多自由，涉及传统上我们认为的言论伤害。比如，如果你试图被模型冒犯，模型会让你被冒犯吗？过去我们对此有更严格的护栏。但我认为模型对齐的一部分是遵循用户希望模型做的事情，在社会决定的广泛范围内。所以如果你要求模型描绘一堆暴力或强化某种刻板印象，是否应该这样做是个问题。我们采取了更宽松的立场。在某个点上，这开始与现实世界的伤害相互作用，我们必须弄清楚如何划定界限，但会有公司说‘好吧，我们听到了社会的反馈。人们真的不希望模型以他们认为不合理的方式审查他们’。这是一场公平的安全协商。

I'm not going to say. And I think that there's really deep care to get this right. I think the caricature of this as just like this crazy race or sprint or whatever, misses the nuance of people are trying to put out models quickly and make great products for people. But people feel the impact of this so incredibly that ... you know, I think if you could go sit in a meeting in OpenAI or other companies, you'd be like, oh, these people are really kind of caring about this. Now we did make a change recently to how we think about one part of what's traditionally been understood as safety. Which is, with our new image model, we've given users much more freedom on what we would traditionally think about as speech harms. You know, if you try to get offended by the model, will the model let you be offended? And in the past, we've had much tighter guardrails on this. But I think part of model alignment is following what the user of a model wants it to do within the very broad bounds of what society decides. So if you ask the model to depict a bunch of violence or something like that or to sort of reinforce some stereotype, there's a question of whether or not it should do that. And we're taking a much more permissive stance. There's a place where that starts to interact with real-world harms that we have to figure out how to draw the line for, but, you know, I think there will be cases where a company says, OK, we've heard the feedback from society. People really don't want models to censor them in ways that they don't think make sense. That's a fair safety negotiation.

但就集体信念问题而言，解决方案是把人们聚集在一起，达成不同的协议。如果有一群人，比如在这里或世界其他地方，愿意举办一个由最优秀的伦理学家、技术专家组成的小型峰会，你和你的同行们试图破解全球一致的安全线，你愿意参加吗？你会敦促你的同事来吗？

But to the extent that this is a collective, a problem of collective belief, the solution to those kinds of problems is to bring people together and meet at one point and make a different agreement. If there was a group of people, say, here or out there in the world who were willing to host a summit of the best ethicists, technologists, but not too many people, small, and you and your peers to try to crack what agreed safety lines could be across the world, would you be willing to attend? Would you urge your colleagues to come?

当然，但我更感兴趣的是我们数亿用户整体的想法。我认为历史上很多决定都是在小型精英峰会上做出的。AI 的一个很酷的新特点是，我们的 AI 可以和地球上的每个人对话，我们可以了解每个人想要的集体价值偏好，而不是让一群被社会认可的人坐在房间里做决定，我觉得这很酷。你会看到我们在这方面做得更多。当我们犯错时，是因为房间里的精英对图像生成护栏的看法与人们实际想要的不同，而且我们无法指出现实世界的伤害，所以我们做出了改变。我为此感到自豪。

Of course, but I'm much more interested in what our hundreds of millions of users want as a whole. I think a lot of the room has historically been decided in small elite summits. One of the cool new things about AI is our AI can talk to everybody on Earth, and we can learn the collective value preference of what everybody wants, rather than have a bunch of people who are, like, blessed by society to sit in a room and make these decisions, I think that's very cool. And I think you will see us do more in that direction. And when we have gotten things wrong, because the elites in the room had a different opinion about what people wanted for the guardrails on image-gen than what people actually wanted, and we couldn't point to real-world harm, so we made that change. I'm proud of that.

数亿人的行为导致意外后果的历史很长。

There is a long track record of unintended consequences coming out of the actions of hundreds of millions of people.

100 个人在房间里做决定也一样。

Also 100 people in a room making a decision.

而且数亿人无法掌控，他们不一定能看到下一步会带来什么。

And the hundreds of millions of people don't have control over, they don't necessarily see what the next step could lead to.

我希望——这完全准确，完全正确——我希望 AI 能帮助我们变得更明智，做出更好的决策，能与我们对话。如果我们说，嘿，我想要某样东西，而不是像人群那样炒作，AI 可以说，嘿，我完全理解你想要什么。如果这是你在这段对话结束时想要的，你在掌控，你选择。但你是否考虑过从这个人的角度或对这件事的影响？我认为 AI 可以帮助我们变得更明智，做出比以往更好的集体治理决策。

I am hopeful that -- that is totally accurate and totally right -- I am hopeful that AI can help us be wiser, make better decisions, can talk to us, and if we say, hey, I want thing X, you know, rather than like, the crowds spin that up, AI can say, hey, totally understand that's what you want. If that's what you want at the end of this conversation, you're in control, you pick. But have you considered it from this person's perspective or the impact it will have on this? I think AI can help us be wiser and make better collective governance decisions than we could before.

我们时间到了。Sam，让你做最后总结。你认为，综合考虑，你的儿子会成长在什么样的世界里？

We're out of time. Sam, I'll give you the last word. What kind of world do you believe, all things considered, your son will grow up into?

我记得——那是很久以前了，我不知道第一台 iPad 是什么时候出来的。大概 15 年前，差不多吧？我记得当时看了一个 YouTube 视频，一个小孩坐在医生办公室的候诊室里，有一本杂志，就是那种旧的、有光泽封面的杂志，小孩把手放在上面，这样划，有点生气。对那个小孩来说，那就像一台坏了的 iPad。他或她从未想过一个没有触摸屏的世界。而对所有观看的成年人来说，这很神奇，因为它太新了，太棒了，是个奇迹。当然，杂志才是世界运作的方式。我的孩子，希望我的孩子们永远不会比 AI 更聪明。他们永远不会成长在一个产品和服务不是极其智能、极其强大的世界里。他们永远不会成长在一个电脑不能理解你的世界里。而且，对于你能想象的任何定义，它都能做到。这将是一个物质极大丰富的世界。这将是一个变化速度极快、令人惊叹的新事物不断涌现的世界。这将是一个个人能力、影响力等远超今天的世界。我希望我的孩子和你们所有人的孩子回头看我们时，会带着一些怜悯和怀旧，说：“他们过着多么可怕的生活。他们太受限了。世界太糟糕了。”我认为这很棒。（掌声）

I remember -- it's so long ago now, I don't know when the first iPad came out. Is it like 15 years, something like that? I remember watching a YouTube video at the time, of like a little toddler sitting in a doctor's office waiting room or something, and there was a magazine, like one of those old, you know, glossy-cover magazines, and the toddler had his hand on it and was going like this and kind of angry. And to that toddler, it was like a broken iPad. And he never, she never thought of a world that didn't have, you know, touch screens in them. And to all the adults watching this, it was this amazing thing because it was like it's so new, it's so amazing, it's a miracle. Of course, you know, magazines are the way the world works. My kid, my kids hopefully, will never be smarter than AI. They will never grow up in a world where products and services are not incredibly smart, incredibly capable. They will never grow up in a world where computers don't just kind of understand you. And do, you know, for some definition of whatever you can imagine, whatever you can imagine. It'll be a world of incredible material abundance. It'll be a world where the rate of change is incredibly fast and amazing new things are happening. And it’ll be a world where, like individual ... ability, impact, whatever, is just so far beyond what a person can do today. I hope that my kids and all of your kids will look back at us with some like pity and nostalgia and be like, "They lived such horrible lives. They were so limited. The world sucked so much." I think that's great. (Applause)

你建造的一切令人难以置信。真的，太不可思议了。我认为在未来几年，你将面临一些最大的机遇、最大的道德挑战、最大的决策，几乎是历史上任何人类都未曾有过的。你应该知道，这里的每个人都会为你加油，希望你做正确的事。

It's incredible what you've built. It really is, it's unbelievable. I think over the next few years, you're going to have some of the biggest opportunities, the biggest moral challenges, the biggest decisions to make of perhaps any human in history, pretty much. You should know that everyone here will be cheering you on to do the right thing.

我们会尽最大努力，非常感谢。

We will do our best, thank you very much.

感谢你来到 TED。（掌声）谢谢。

Thank you for coming to TED. (Applause) Thank you.

非常感谢。

Thank you very much.