Simile 创始人 Joon Sung Park：以 85% 以上准确率预测人类行为

你正在收听 Gradient Descent，一档关于让机器学习在现实世界中发挥作用的节目，我是你的主持人 Lukas Biewald。好的，今天和我对话的是 Joon Park，他是用生成式 AI 模拟人类行为的世界级专家之一。他也是 Simile 的 CEO，这是一家年轻的公司，在帮助其他公司模拟人类行为方面做得非常出色。他说在这次采访前模拟了我，我希望我能比我的数字孪生表现得更好。希望你喜欢这次对话。

You're listening to Gradient Descent, a show about making machine learning work in the real world, and I'm your host, Lukas Biewald. All right. Today, I'm talking with Joon Park, who is one of the world's experts on simulating humans with generative AI. He's also the CEO of Simile, which is a young company that's doing phenomenally well helping other companies simulate human behavior. He says that he simulated me before this interview, and I'm hoping I do better than my digital twin. And I hope you enjoy this conversation.

好的，Joon，我非常兴奋能和你聊聊你的公司 Simile，但在那之前，我想先谈谈你的博士论文，它是我一生中读过的最有趣的论文之一。你想告诉我们你做了什么吗？

All right, Joon, I'm super excited to talk to you about your company, Simile, but before we get into that, I wanted to talk to you about your PhD dissertation, which is like one of the most fun dissertations I've ever read in my life. Do you want to tell us what you did?

是的，当然。感谢邀请我。非常兴奋能来到这里。所以，我在 2020 年开始读博士，最终为博士学位所做的论文是关于创建生成式智能体模拟人类的想法。当时的背景是，当我开始读博时，GPT-3 即将问世。它还没有完全出现。我们能用的是 GPT-2。它非常不稳定，但你能隐约看到这个模型有能力做一些非常令人兴奋的事情。所以，在我的第一年，我们和许多斯坦福同事一起写了一篇论文，题为《基础模型的机会与风险》，由我现在在 Simile 的联合创始人 Percy Liang 领导。在那里，我们有很多不同的假设，关于我们认为这个模型会走向何方，以及真正的杀手级应用会是什么。我们早期的一个观察是，这些模型的独特之处在于它们在训练数据中感知或观察到了大量人类数据，如果你从正确的角度去探究，你实际上可以提取出很多真实的人类行为。所以，在我的第二年和第三年，我们开始围绕这个想法进行博士论文研究，最终在 2023 年写了一篇论文，题为《生成式智能体：人类行为的交互式模拟》。在那篇论文中，我们创建了一个小游戏城镇，里面有 25 个智能体生活。这些智能体会在早上醒来，做他们的日常事务，去上班，建立关系，做所有这些了不起的事情，基本上创建了一个看起来很像我们的城镇。他们实际上会聚在一起甚至举办派对，就像自组织一样。所以，我觉得那非常有趣。这最终成为我工作的核心 backbone，我们有一个非常独特的机会，首次使用这些模型创建真正可泛化的智能体，它们可以像人类一样生活和行为。所以，在那项工作之后，我们实际上收到了很多主动的兴趣。最初，实际上来自科学界，来自社会科学家、心理学家、行为科学家，他们非常热衷于进行实验。他们都认为，“嘿，如果我们能创建这样的模拟，有很多我们很想在现实生活中进行的实验，但我们要么没有资源，要么根本没有能力进行。”但如果我们能模拟，那将非常有趣。

Yeah, for sure. And thanks for having me. Really excited to be here. So, I started my PhD back in 2020, and the dissertation that I ended up working on for my doctoral degree was this idea of creating generative agent simulations of people. The context here was when I started my PhD, that was when GPT-3 was about to come out. It wasn't quite there yet. We had access to GPT-2. It was extremely janky, but you could sort of see the glimpse of this model being able to do something really exciting. So, during my first year, we ended up writing this paper called Opportunities and Risks of Foundation Models alongside a large number of Stanford colleagues, led by one of my co-founders now at Simile, Percy Liang. And there, we had a bunch of different hypotheses in terms of where we thought this model was going to go, but what the killer application was actually going to be. One of the early observations we made was these models were unique in that they perceived or they observed so much of human data in its training data that if you sort of poke at it at the right angle, you can actually extract a lot of realistic human behavior. So, we started to play around that idea during my second and third year into my doctoral dissertation or degree, and we ended up writing a paper called Generative Agents: Interactive Simulacra of Human Behavior in 2023. That was a paper where we created this small game town with 25 agents living in it, and these agents would wake up in the morning, do their routines, go to work, have relationships, do all that amazing things, and basically create a town that looks a lot like ours. And they would actually come together to even have parties, like self-organize. So, I thought that was fantastically interesting. So, that ended up becoming the core backbone of my work where we have this very unique opportunity to use these models to create agents for the first time that's actually generalizable, that can actually live and behave like humans. So, after that work, we actually got a lot of inbound interest. Initially, actually from the scientific community, from social scientists, psychologists, behavior scientists, who are very much into running experiments. They all thought, "Hey, if we can create simulations like this, there's so many experiments that we would have loved to run in real life that we either don't have resource for or we just do not have the ability to run." But if we can simulate that, that'll be fantastically interesting.

好的，但在你进入实际应用之前，构建一个包含所有这些小角色的小世界有趣吗？就像玩娃娃一样。

Okay, but before you get into practical applications, was it fun to build like a little world with all these little characters in it? Like it almost feels like playing with dolls or something.

感觉很像那样。事实上，我从小就是科幻迷。我小时候经常看《数码宝贝》、《宝可梦》这类游戏和电视节目。感觉有点像创建那些游戏或那些系列中描述的世界。显然，我也是这些大型模拟游戏的粉丝，比如《模拟人生》之类的。所以，基本上就是把那种体验带入现实生活，我们可以实际塑造和玩耍。这真的非常值得。

It felt a lot like that. In fact, I grew up as a fan of amazing science fiction. And there's actually this TV series I used to watch like Digimon, Pokémon, these kind of games and TV shows when I was very young. And it felt a little bit like creating those games or those worlds that were described in these series. And obviously, I grew up a fan of these large-scale simulation games as well, like The Sims and these kind of things. So, it basically was bringing that into real life in ways that we can actually morph and play with. And it was actually really worth it.

所以，你有这些语言模型。我想当时它们甚至没有真正的图像生成。那么，你是告诉每个智能体发生了什么，然后问它下一步做什么，还是创建了一个小虚拟世界让它们互相交流？它到底是如何工作的？

So, you have these like language models. And I think back then, they didn't even really do image generation. So, are you kind of telling them each one what's going on and then asking it what it does next or did you create like a little virtual world for them to talk to each other? How did it really work?

对。你说得完全正确，当我们做最初的生成式智能体论文时，用的是 GPT-3.5。所以模型非常不稳定。甚至让它像人类一样说话，我不得不做的是实际创建一个提示。那时甚至不是 ChatGPT。就像你在 playground 里玩了很长时间。你会实际制作一个提示，基本上有 HTML 标签，你会说 div class chat，然后 div class comment，或者在 thread 中，这就是你真正推动模型的方式。所以你可以想象这东西有多不稳定。

Right. So, you're exactly right that when we did the original generative agents paper, it was with GPT-3.5. So, the model was extremely janky. Even to get it to talk like humans, what I even had to do was I would actually create a prompt. It wasn't even ChatGPT back then. It was just like, you were literally playing in the playground for the longest time. You would actually craft a prompt that basically has like HTML tags and you would say div class chat and then div class comment or, you know, in the thread and that's how you actually really push the model. So you can imagine like how janky this thing was.

是的，是的。

Yeah, yeah.

但我们最终做的是，我基本上在游戏创建的地图编辑器中创建了这个小型游戏城镇。我基本上在上面叠加，并以此方式编码地图上的不同元素，有点类似于人们实际创建障碍物或智能体不能穿越的地方的方式。

But what we ended up doing was I ended up creating this small game town in basically a map editor for game creation. And I basically overlaid on top of it and this way to encode different elements on this map in a way that's a little bit similar to how people would actually create the blocks or sort of the places that the agents cannot cross.

所以游戏制作者其实一直这么做：你看到的是 3D 或 2D 地图，但底层基本上是一个网格，标明这八个地方是智能体不能穿过或进入的。

So the game makers actually would do this all the time where you see the 3D or 2D map, but under the hood it's basically a grid that basically says these are the eight places where the agent cannot cross or access.

嗯。

Mhm.

我基本上用这个机制来标明冰箱在哪里、你的房子在哪里，然后你就能创建整个小镇。智能体会四处走动，以这种方式感知物体，但随后会从视觉媒介翻译成语言，我们再把它反馈到语言模型的上下文里。

I basically use that mechanism to say this is where the refrigerator is, this is where your house is, and you'd create an entire town. Agents would walk around and perceive the objects in that way, but it would then get translated from the visual medium to language and we just feed back into the context of language models.

所以你会说，比如你在这个小镇的这里，这是你看到的东西。而且你给每个智能体起了名字，对吧？还有性格？有哪些涌现行为发生了？

So you'd say like here's where you are in this town, here's what you're seeing. And you gave each agent names, right? And personalities? And what were some of the emergent behaviors that happened?

对。有很多非常有趣的事情。我们给这 25 个智能体起了名字，其中一些其实是以我的朋友、家人和同事命名的。但基本上这些智能体开始做的事情是，我们实际上有一对

Right. So a lot of really interesting things. So we named these 25 agents, some of them actually after my friends and family and colleagues. But basically the sort of things these agents started to do was we actually had a couple

嗯。

Mhm.

在模拟开始时他们并不是情侣，但最终发生的是他们彼此都有好感。

where who at the beginning of the simulation they were not couples, but basically what ended up happening was they both had a crush on each other.

嗯哼。

Uh-huh.

所以他们在模拟过程中可能做的事情，你知道，就是一个智能体会约另一个智能体出去。

So the kind of things they might do throughout the simulation actually is you know one agent would ask the other agent out.

嗯。

Mhm.

而且恰好有一场情人节派对正在进行。模拟被设定在 2 月 13 日开始。所以这个小镇上有一位咖啡馆老板，她计划在咖啡馆举办一场情人节派对。于是她会四处邀请她的朋友和顾客来参加。而我们其中一个智能体约了另一个智能体去那个派对。所以这些就是我们看到的一些涌现行为。令人惊叹的是，我们没有计划任何这些行为。我们只是给了他们一串人设或一些关于他们是谁的描述。剩下的就顺其自然地展开了。

And there happened to be a Valentine's Day party going on. The simulation was set to start February 13th. So we had a cafe owner in this town who was planning to have a Valentine's Day party at a cafe. So she would actually ask around her friends and her customers to come join her. And we had one of these agents ask another agent out to that party. So those are some of the emergent behaviors that we saw. And the amazing part here was we didn't plan for any of these behaviors. We just gave them a string persona or some description of who they are. And the rest sort of unfolded as they wanted to.

哇。你开始对这些智能体产生感情了吗？

Wow. Did you start to get attached to these agents?

你知道，我显然明白这些只是游戏中的角色，但有点像我能想象小说家会对他们的角色产生感情。我确实对其中一些角色产生了感情。当我们创办公司时，同样地，我们会议室的名字实际上就是以这些角色命名的。

You know, I obviously understood, obviously, that these are merely characters in these games, but a little bit like the way that I can imagine a novelist would get attached to their characters. I certainly did get attached to some of these characters. When we started the company, similarly, the names of our rooms for meetings actually were named after these characters.

酷。

Cool.

所以我们主会议室或董事会会议室实际上叫 Isabella。

So our main conference room or the board room was actually Isabella.

嗯。

Mhm.

她就是那个咖啡馆老板。当我做关于这个项目的演讲时，她有时也会作为我的伴侣出现。

She was the cafe owner. And she sometimes made appearance also as my plus one when I give talks about this project.

不会吧。

No.

所以，非常投入。

So, very much attached.

那你是怎么把这个包装成研究的？

And how did you package this as research?

对。所以一开始我们确实有些挣扎。我们早在 2021 年就有了创建这个游戏小镇的想法。但我们面临的挑战是，我们认为如果能创造这样一个世界会非常引人入胜，但我们就是无法很好地表达为什么我们要这样做。因为那时 GPT 还处于非常早期的阶段。所以我们不知道这是否是这个领域应该发展的正确方向。我们也看不清用例。

Right. So initially, we actually had some struggle. We actually had the idea of creating this game town all the way back in 2021. But the challenge that we had was we thought it would be really compelling if we can create a world like this, but we just couldn't quite express why we would want to do this. Because it was very early days of GPT. So we didn't know whether this is the right way this field ought to progress. And we also couldn't quite see the use cases.

嗯。

Mhm.

对吧？所以我们创造了这个神奇的游戏小镇，而我许多同事一开始想，‘你为什么要做这个？你为什么要研究这个？这只是一个游戏吗？’所以我们在 2021、2022 年实际做的是，我们有一篇论文，是 Smallville 生成式智能体论文的前身，叫做 Social Simulacra。这里的想法基本上是，至少动机是，我们对这个游戏小镇感到兴奋，但在我们真正达到那一步之前，我们想至少向自己证明这是值得做的事情，如果我们能做得非常非常好，就会产生影响。所以这里的想法是，当你在设计一个社交系统时，极难预见可能发生的一切。就像所有涌现现象，所以你想象你创建了一个新的社交媒体网站，或者你想象你有一个新的社区，所有他们可能做的好事，以及所有他们可能做的令人惊讶和负面的事情，所有反社会行为，都很难预见。所以我们想，好吧，如果我们能创建一个 subreddit 的模拟呢？所以这基本上是 2021、2022 年版本的 Mo Bug。我们认为这可能让我们有能力原型化我们设计社交算法或社交媒体的方式，这样，你知道，过去有很多关于人们行为不端或出现这些非常反社会行为的问题，方式可能具有攻击性，剥夺了某些人的权力。我们想，嘿，如果我们能给设计师提供工具呢？我的意思是，最终，设计师可以设计他们想要的任何社交媒体或任何社区，但至少让这些选择是有意为之的。他们会向你展示你设计的影响，设计师完全可以决定他们想怎么处理，但我们能不能真正模拟那个结果？所以这最终成为了一篇非常有趣的论文，那是我们的第一篇论文，也是我们在该领域看到的第一篇使用生成式 AI 创建真实社区、也创建真实人设的论文。当时‘人设’这个词在这个语境中也不存在，但那时我们引入了创建个人数字智能体的想法。这引出了生成式智能体论文，我们现在获得了信心，这是一个有趣的想法。我们可以用这个创建真正的应用，如果我们能做好这类模拟，那这真的会很有价值。所以那时我们想，那么我们能利用这个想法创造的最疯狂的东西是什么？我自己和研究小组喜欢这样思考的方式是玩时间机器游戏。如果我们快进 10 年到未来，我们能做的最雄心勃勃的事情是什么？然后试着现在去做。对我们来说，那实际上是，嘿，我们创建了一个 subreddit。我们现在能创建一个完整的世界吗？所以就是这样来的，但即使直到那个特定项目的生命周期结束，实际上我内心也有很多不安全感，我们认为这是一个非常有趣、引人入胜的想法，但我无法很好地表达为什么这如此重要，除了它吸引我。然后这是一个活起来的小镇。这很有趣。但这是研究吗？不太说得清。

Right? So we create this amazing game town, and many of my colleagues initially thought, 'Well, why are you doing this? Why are you working on this? Is it just a game?' So what we actually ended up doing back in 2021, 2022 was actually we had a paper that was precursor to the Smallville generative agents paper called Social Simulacra. And the idea here basically was, at least the motivation was, we were excited by this game town, but before we quite get there, we wanted to at least prove to ourselves that this is something that is worthwhile, that this is something that if we can do this really, really well, that will have impact. So the idea here was, when you are designing a social system, it is extremely hard to foresee everything that might happen. Like all the emerging phenomenon, so you imagine you create a new social media site, or you might imagine you have a new community of people, all the good things they might do, but also all the surprising and negative things they might do, all the antisocial behavior, it's very hard to foresee. So we thought, well, what if we can create a simulation of a subreddit? So this is basically 2021, 2022 version of Mo Bug. We thought that might give us the ability to prototype the way we might design the social algorithms or the social media, so that, you know, in the past, there was a lot of issue around people misbehaving or having these very antisocial behaviors in ways that is perhaps aggressive, that disempowers certain people. We thought, hey, what if we can just give the designers tools? I mean that in the end, it is up to designers to design whatever social media or whatever community they want to design, but at least make those choices to be intentional. They will show you the implications of your design, and it's totally up to the designer in terms of what they want to do with that, but can we actually simulate that outcome? So that ended up actually becoming a very interesting paper, and that's the first paper that we had, and also that we've seen in the field, that used generative AI to create a real community, but also real personas. That the term persona didn't quite exist in this context either, but that's when we introduced the idea of creating personal digital agents. That led to the generative agents paper where we now gained the confidence that this is an interesting idea. There are real applications that we can create with this that if we can do these kind of simulations well, then this is really going to be valuable. So at that point we thought then what is the craziest thing that we can create using this idea and the way myself and we in the research group like to think about this was playing the time machine game. That if we were to fast forward 10 years into the future, what is the most ambitious thing that we could have done? And try to do it now. And for us that actually was hey, we created a subreddit. Can we now create an entire world? So that's how it came about, but even until the end of that particular project's life cycle, there actually was a lot of insecurity something in myself and we think this is a really interesting compelling idea, but I couldn't quite express why this was so important other than it compelled me. Then here's a town that is coming alive. This is interesting. But is this research? Couldn't quite tell.

这确实看起来引人入胜。那种东西一直对你很有吸引力吗？我的意思是，你看到很多孩子这样玩，想象人们的世界。

It does seem compelling. What is it is that kind of thing always been compelling to you? I mean you watch a lot of kids kind of play in this way and imagine worlds of people.

这是你一直感兴趣的东西吗？这类游乐场式的模拟？

Is that something that's always interested you? These kind of playground simulations?

你知道，小说——其实我从小就是个艺术家。我的专业学位最初是工作室艺术，我是个油画画家。我觉得，在某种程度上，创建这样的模拟感觉很像你在创建一个模拟，或者你在画一幅我们所生活社会的肖像。对吧？因为肖像归根结底是对我们所感知的世界真相的表达。模拟很像那样。这些智能体代表真实的人，它们的互动代表了人们在这种环境中可能实际如何行为。所以，这吸引了我。

You know, fiction—I in fact grew up as an artist. My professional degree actually initially was in studio art and I was an oil painter. And what I felt was, to some extent, this creating simulation like this feels a lot like you're creating a simulation or you're creating a portrait of the society that we live in. Right? Because a portrait at the end of the day is an expression of what we perceive to be the truth in the world that we live in. Simulation is a lot like that. These agents represent real people and their interaction represent how people might actually behave in these kind of environments. So, that compelled me.

有趣。你怎么知道你的模拟好不好？

Interesting. How do you know if your simulations are good or not?

嗯。这是个好问题。生成式智能体论文之后，最终发生的事情就是为了深入回答这个问题。最初，当我们发表这篇论文时，我们的评估，我觉得挺有意思的。当时的评估，我们从这里开始，就是我们会采访这些智能体。所以，我实际上会把 25 个智能体每个都拉出来，让它们坐在命令行界面里，说：“我要问你问题。告诉我你是谁。描述你爱的人。告诉我你有什么爱好。告诉我描述你的一天。”我基本上会请真实的人类作者也来手写这些。然后我们会比较，给定这个人，这些智能体说的合理吗、现实吗？我们就是这样开始的。所以，在 2023 年，在这个游戏小镇里，我们称之为可信度评估。当我们看它们时，它们说的和做的在很大程度上可信吗？它们基本上通过行为图灵测试了吗？

Mhm. So, this is a great question. So, what ended up happening after the generative agents paper was to really answer this question in a deep way. Originally, when we published this paper, our evaluation, I thought was sort of interesting. The evaluation back then, where we start from, was we would interview these agents. So, I would actually pull each of the 25 agents out and sit them down in this command line interface and said, "I'm going to ask you questions. Tell me who you are. Describe who you love. Tell me what hobbies you have. Tell me describe to me your days." And I would basically ask real human authors to also come in and write them by hand. And we'd actually compare, given this person, is what these agents saying reasonable, realistic? That's how we started. So, in 2023, in this game town, we call this believability evaluation. That when we look at them, is what they're saying and what they're doing largely believable? Are they basically passing behavior Turing test?

嗯。

Mhm.

但后来，当我们想超越这一点时，我们想表明这些不仅可信，而且准确。这基本上意味着我们需要在世界上有真实的地面真相，并看看这些智能体的行为是否真的与地面真相匹配。所以，在接下来的一年里，我们实际上联系了很多人，把他们带到实验室，再次以采访的形式收集他们的数据。但这次不是采访智能体，而是采访真实的人，我们基本上问他们这样的问题：“嘿，告诉我你的人生故事。”嗯，一直到实际进行一些行为研究，以便了解他们的行为倾向，用这些来训练这些智能体，让人们离开几周再回来，当他们回来时，我们基本上问很多行为问题和行为研究，并试图让他们的智能体或基本上他们自己的智能体版本预测他们的行为。

But then, when we wanted to go beyond that, we wanted to show that these are not just believable, but that they're accurate. What that basically meant was that we needed to have real ground truth in the world and see whether the behaviors of these agents actually match the ground truth. So, we actually ended up in the subsequent year, we ended up actually reaching out to a lot of people, brought them to a lab, and collected their data actually in the form of interview again. but this time instead of interviewing the agents, we interviewed real people, and we basically asked them questions like, "Hey, tell me the story of your life." Uh all the way to actually ran some behavior studies so we can get to know their behavioral tendencies, use that to train these agents, send the people away for a couple of weeks and bring them back, and when they're back, we basically ask them a lot of behavior questions and behavior studies, and try to have their agents or basically their agent version of themselves predict their behavior.

有趣。那么，什么是行为问题？

Interesting. So, what's like a behavior question?

所以，在这项特定研究中，我们基本上有四类研究。嗯，一类是我们称之为慷慨的社会调查。这是一项大型的关于人们生活观点的调查，社会科学家会将其视为调查的典型例子。嗯，然后是人格测试。我们进行了大五人格测试。另一个是经济行为游戏。所以，这些包括诸如囚徒困境之类的行为游戏，这实际上是我们进行的测试之一，但我们基本上有相当多的这类行为游戏。最后一个是对照试验。所以，社会科学家经常设计随机对照试验，基本上像是 AB 测试的稍复杂版本，他们基本上有相同的实验设置但不同条件，你在人们身上运行，看看他们的行为在实验中是否改变。我们实际上采取了大量发表在真实科学论文上的行为研究，比如在 PNAS、Nature 等上，把它们带到实验室，用我们的参与者重现它们。这些就是验证集，我们基本上让我们的智能体预测我们的真实参与者对这些研究会如何反应。这就是在社区和科学领域中，我们首次展示我们可以以 85% 以上的准确率预测人们的行为。这就是这个验证方案的基础，也是类似的基础。

So, in this particular study, we basically had four categories of studies. Uh one was what we call the generous uh social survey. This is a large uh survey of people's viewpoints in their lives that social scientists will rely on as sort of the canonical example of the survey. Uh and then there was personality test. We ran big five personality test. Another one uh was economic behavior games. So, these included things like uh literally uh prisoner's dilemma, ask uh kind of behavior game that actually was one of the tests that we ran, but we basically had uh good number of these behavior games. The last one uh was randomized control trials. So, social scientists would often design randomized control trials basically sort of as a little more complex version of AB testing where they basically have same experimental setup with different conditions, you run it on people and see whether their behavior changes in that experiment. We actually took a large number of behavior studies uh that were published in real science papers uh like in PNAS, Nature, and so forth, brought them to lab and recreated them with our participants. And these were the set of validations, and we would basically have our agents predict how our real participants would react to these studies. And this is where for the first time in the community and in the scientific field, we show that we can actually predict people's behavior with 85% plus accuracy. And that's what laid the groundwork for this validation scheme, but also the foundation for similarly.

85% 的准确率意味着什么？比如这些是 AB 问题吗，基于此会是 50% 还是什么？很难理解 85% 真正意味着什么。那好吗？

And what does 85% accuracy mean? Like was it are these AB questions where based on it'd be 50% or like what what would It's hard to understand what 85% really means. Is that good?

所以，我们看待这个问题的方式是，人们有——在我们问的特定问题中，有很多离散选择问题。在那些问题中，我们基本上看我们是否完全答对了。其中一些问题会有五个选项、10 个选项。嗯，对于像行为研究或随机对照试验之类的一些东西，我们也会测量效应量。我们基本上会计算相关性。所以，严格来说那不是准确率度量，但我们会运行皮尔逊相关之类的东西。当你进行相关性研究时，如果皮尔逊相关大于 0.6，那是稳健的相关。如果大于 0.75 到 0.8，那是非常强的相关。我们最终看到的是，在这些上，相关性方面，我们会看到 0.7 到 0.75 的范围。所以，强相关。准确率约 85%。现在，这里重要的是要注意，我们发现特别有趣的是，当你问个人层面的问题时，甚至人类自己在被问同样问题时也会改变他们的回答。

So, uh the way we look at this is people have So, in in particular questions that we ask, there are many discrete choice questions. In those questions, we're basically seeing uh whether we got the answer exactly right. And some of these questions would have five response uh five options, 10 options. Uh and for some of something like um behavior studies uh or randomized control trials, we'll also measure the effect size. And we'll basically calculate the correlation. So, there technically that's not accuracy measure, but we would run things like Pearson correlation. And when you run correlation study, if the Pearson correlation is greater than 0.6, that's robust correlation. If it's greater than 0.75 to uh 0.8, that's very strong correlation. What we ended up seeing was correlation-wise on these, we would see something in the range of 0.7 to 75. So, strong correlation. Accuracy about 85%. Now, what is important to note here that we found to be particularly interesting was when you ask questions at the level of individuals, even humans themselves change their responses when they're asked the same question.

完全正确。

Totally.

嗯，所以这里实际上存在某种准确率的上限。对吧？例如，如果你能以 100% 的准确率预测人们的回答，那实际上很奇怪。

Uh so, there was actually some kind of ceiling to accuracy here. Right? That it is it is actually strange, for instance, if you can predict people's responses at 100% accuracy.

嗯。

Mhm.

甚至我会说 95% 以上的准确率。因为当我们进行人类受试者研究时，他们对我们进行的研究的回答大约有 13% 到 14% 的次数会变化。所以，准确率的上限实际上是 86% 到 89%。嗯，在许多这些问题中。所以，当我们说 85% 或 75% 到 85% 以上的范围时，那实际上对我们来说是一个非常稳健的范围。这基本上意味着你越来越接近与人类受试者持平。现在，更重要的是，你能在你感兴趣的所有人群中复制这种预测强度吗？这些模型显然最初是在大量网络数据上训练的。所以，对于像我这样的人，或者也许像我们俩这样的人，对于在网络上有良好代表的人群，这些模型在预测他们的行为方面做得相当稳健。

Or even I would say 95% plus accuracy. Because when we run human subject studies, their answers on the studies that we run changes about 13% to 14% of times. So, the ceiling of accuracy really is 86% to 89%. Uh in most of a lot of these questions. So, when we say 85% or 75% to 85% plus range, that's actually a very robust range for us to be. That basically says you're getting closer to reaching parity with human subjects. Now, then what matters more is can you replicate that strength in prediction across all population of your interest? These models obviously are initially trained on a lot of data from the web. So, for people like myself, uh or perhaps people like uh both of us here, for well represented on the web population, these models do a pretty robust job at predicting their behaviors.

但在你深入之前，你能做出哪些预测？比如，你能预测我在囚徒困境中更可能合作还是背叛吗？或者有哪些预测让你印象深刻，是你原本以为模型做不到的？

But before you get into this, what kinds of predictions can you make? Like, can you predict whether I'm likely to cooperate or defect in prisoner's dilemma? Or what were the predictions that really impressed you that you wouldn't have thought the model would be able to do?

实际上最重要的预测是这些模型能够预测随机对照试验。这对我 and 我的同事来说是最重要的事情。思考这个问题的方式以及原因在于，在科学中，我们显然做了很多观察性研究来理解世界的状态。

The predictions that actually matter the most was the fact that these models could predict the randomized control trials. That actually was the thing that mattered the most to me and to my colleagues. The way to think about this and the reason why that's the case is in science, obviously we do a lot of observational studies to understand the state of the world.

嗯。

Mhm.

但通常，我们真正问的问题是，我们如何帮助设计系统，让世界变得更美好？

But oftentimes, really the questions we're asking is how can we help design systems that would make the world a better place?

嗯。

Mhm.

这不仅仅是预测未来或发现相关性，而是关于因果机制，并通过干预实际塑造未来。所以，对我们来说，重大的科学问题是，它真的能做 A/B 测试吗？它真的能做随机对照试验吗？在那里，你必须注意的是，不仅仅是我們能否预测，而是我们能否实际预测不同干预的结果或效果？我们能在那些情境下达到 0.75 到 0.8 的相关性，这非常强大。

That is not about simply predicting the future or finding the correlation, but that is about causal mechanism and actually shaping the future through interventions. So, the big scientific question for us actually was, well, can it actually do AB testing? Can it actually do randomized control trials? And there, the thing that you have to watch out for is not merely can we predict, but can we actually predict the outcome or the effect of different interventions? The fact that we could get in those contexts to correlation of 0.75 to 0.8 was very powerful.

嗯。但我想象，在这些科学研究中，有人提出假设，认为会有相关性。

Mhm. But I imagine like with these scientific studies, someone's making a hypothesis that they think there's going to be a correlation.

是的。

Yeah.

这实际上比研究人员更准确吗？还是比预测某人会做什么更准确？

Is this actually more accurate than the researchers or is it more accurate than like say if you were predicting what somebody would do?

在某些情况下，是的。在某些情况下，至少社会科学家在生成假设时，他们有强烈的直觉。这通常是他们运行那些假设的原因。但通常我们的直觉会受到检验。

So in some cases, yes. In some cases at least a social scientist when they generate hypotheses, they have strong intuition. And that's often the reason why they run those hypotheses. But oftentimes our intuitions are tested.

嗯。

Mhm.

显然，如果我的直觉总是对的，我作为混合研究研究员会更 prolific。

Obviously if my intuition was always right, I would have been even more prolific as a mixed study researcher.

当然。

Sure.

所以这些研究中有真正的惊喜。模拟的力量部分在于我们可以模拟人，我会非常具体。我们经常感到惊讶的原因是因为人们在野外的行为和态度与你自己作为研究员的不同。我以某些方式行事。我有某些价值观和行为倾向。但它可能实际上与思想不同的人不匹配。所以这就是我有误解的地方。模拟的力量在于它能够大规模地代表人们。所以它基本上是一种将我对人们是什么样的直觉与非常多样化和代表性的人群现实连接起来的方式。

So there's real surprises in these studies. And the power of simulation in part is we can simulate people and I'll be very concrete. The reason why we're often surprised is because people's behavior and attitude in the wild is different from your own, the researcher. I behave in certain ways. I have certain values and behavior tendencies. But it might not actually match with people who are different in mind. So that's where I have the misconception. Simulation, the power is it's the power to represent people at scale. So it's basically a way to bridge my intuition on what people are like to the reality of a very diverse and representative set of population.

你能给我一些你能做出的具体预测的例子吗？

Can you give me some examples of concrete predictions that you're able to make?

是的。所以如果你今天看 Simile，我们通常带来的预测类型是概念测试、信息测试，然后进入产品测试。例如，你想推出一个新的，甚至可以是你有一个新的移动应用程序，人们会在那里购买物品或采取某些行动，你可以实际问智能体：“嘿，你能用这个几分钟，告诉我们你对这个的看法吗？你会在哪里卡住？”

Yep. So if you were to look at Simile today, the kind of prediction that we usually come in with and help is we do concept testing, message testing and then this goes into product testing. So for instance, you want to launch a new it can even be something like you have a new mobile application where people are going to purchase items or they're going to take certain actions, you can actually ask agents, "Hey, can you use this for a few minutes and tell us what you think about this? Where would you get stuck?"

嗯。

Mhm.

你想看到什么？一直到你喜欢这里的配色方案吗？这引起共鸣还是太累人？这些是我们帮助回答的非常好的模拟问题。然后随着我们深入，我们实际上会做这样的事情：“嘿，现在我们尝试改变颜色。我们尝试改变这个广告这个产品的措辞。这更引起共鸣吗？”在某些情况下，尤其是当我们更接近业务需求时，这会增加留存率吗？这会增加这个业务的销售额吗？是我们将回答的问题类型。

What do you want to see? All the way down to do you like the color scheme here? And does this resonate or is this too tiring? These are really good simulation questions that we help answer. And then as we get deeper into this, we would actually do things like, "Hey, now we try changing the color. We try changing the wording of this ad this product. Does this resonate more?" And in some cases, especially as we get closer to the business needs, does this increase the retention? Does this increase the sales of this business? Is the kind of questions that we'll be answering.

我明白了。我想你将要说的一个重要问题是，这些 LLM 主要是在网络数据上训练的，而网络数据是人类分布相当偏斜的。我们可能被过度代表，而很多人被代表不足。你如何管理这些效应？

I see. And I think you were going to say an important question, which is that these LLMs are trained mostly on web data, which is a pretty skewed distribution of humans. We're probably very overrepresented and a lot of people are underrepresented. How do you manage for those effects?

对。我的意思是，我们有时我觉得我永久在线，所以我肯定被这些模型很好地代表了。但这就是 Simile 训练的模型与实验室训练的模型根本不同的地方。所以 Simile 是一家训练自己的人类行为基础模型的公司。思考这个问题的方式是，像 ChatGPT 或 Claude 这样的模型是推理机器。它们从专家那里获取大量推理数据。这些专家可能是了不起的程序员、数学家、自然科学家。它们基本上试图复制这些专家解决问题的方式。这些问题通常也有非常具体的形式和什么应该被认为是正确答案。它非常客观。另一方面，Simile 创建的模型根本不在乎在任何这些任务上成为超人。我们所做的就是实际去从真实的人那里收集数据。所以，我们现在与大约 40 家不同的供应商合作，但我们的一个重点合作是我们与盖洛普有战略合作，例如。所以，我们实际上会去，他们有一套了不起的代表性面板。所以，我们实际上会去与所有人口统计、人群和子人群中的代表性真实人群交谈，试图理解他们的价值观和行为，我们会直接将其放入模型中。模型的目标函数是接受这些并学习在我们感兴趣的整个人群中准确的分布。

Right. I mean, we sometimes I feel like I am permanently online, so I'm definitely well represented by these models. But this is where the model that Simile is training is fundamentally different from the models that are getting trained in the labs. And so Simile is a company that is training its own foundation model issue of human behavior. And the way to think about this is models like ChatGPT or Claude are reasoning machines. They take a lot of reasoning data from experts. These could be amazing programmers, mathematicians, natural scientists. And they basically try to replicate the way these experts solve problems. And these problems also have often times very concrete shape and what should be considered as the right answer. It's very objective. The model that Simile creates, on the other hand, does not care at all about becoming superhuman across any of these tasks. All we do is we actually go, collect data from real people. So, we have partnership with about 40 different vendors now, but one of our highlighted partnership is we have a strategic partnership with Gallup, for instance. So, we would actually go and they have an amazing set of representative panels. So, we would actually go and talk to a representative set of real people across all demographics, populations, and subpopulations and try to understand their values and their behaviors, and we would put that directly into the model. And the objective function of the model is to take that and to learn the distribution that is accurate across the entire population of our interest.

当你说准确时，你是指不是预测特定的事情，而是预测答案的分布？

And when you say accurate, you mean not predicting a specific thing, but predicting the distribution of answers?

所以，我们专门训练两种预测模型。一种是我们称之为群体层面模型。这个模型的目标是获得一群人和环境的描述，并模拟或预测该群体中的人在这种情况下会如何行为的分布。然后，我们训练的另一个模型是我们称之为个体模型。在这里，实际上，目标是实际建模 Lucas 或实际建模 Jane，并说：“在这个背景下，这些个体会做什么？”我们，当我们开始公司时，我们最初从个体模型开始，因为在许多方面，个体模型可以提供某些用例，但此外，个体通常是更大群体的量子单位。所以，如果你能把个体层面的模型做对，那么这相当自然地导向群体层面模型。

So, we do specifically train two kinds of models for prediction. One is what we call the population level model. The goal of this model is to get a description of a group of people and the environment and simulate or predict the distribution of how people of that group would behave in this circumstance. And then, the separate model that we train is what we call the model of individuals. Here, actually, the goal here is to actually model Lucas or to actually model Jane and say, "In this context, what would these individuals do?" We, when we started the company, we originally started with the model of individuals because in many ways, there are certain use cases the models of individuals can provide, but also, the individuals are usually the quantum units of larger populations. So, if you can get the individual level model right then that leads fairly naturally to population level models.

你们完全从零开始训练这些模型吗？

And you train these models entirely from scratch?

呃，我们现在做的是中期训练和后训练。所以不是完全从零开始。我们确实利用开源模型作为起点。呃，但之后有相当大的努力投入到调整参数上，以确保我们能够大规模地代表这些人。

Uh so we do mid and post training right now. So it's not entirely from scratch. So we do leverage open source model as a starting point. Um but there's a fairly large effort that goes into shaping the parameter afterwards to make sure that we can represent these people at scale.

有趣。我想，比如获取对文本的反应可能很容易，但色彩体验，在预训练语言数据集中可能没有很好地体现，会更难找到，或者人们做但不一定谈论的事情，对于你想要的数据类型来说会很难收集。你如何解决这个问题？

Interesting. And I would think it might be easy to get reactions to text, for example, but experiences of color, something that's not as well represented in like a pre-training LN data set would be much harder to find or or things that people do that they don't necessarily talk about would be hard to collect for the kinds of data that you want to How do you account for that?

所以我们实际上收集了很多这样的数据。呃，我们可能实际进行的数据收集是，我们深切关注的不只是理解最终结果或最终产品以及人们关心的最终信息，而是试图理解量子单元。例如，显然来自更视觉艺术背景的人，艺术家会关心的是配色方案，我们深切关注。呃，纹理我们深切关注。整体形式和风格，所以我们是极简主义者还是形式主义者还是极繁主义者？这些风格选择我们非常关心。所以我们首先在真实数据中试图理解的是人们在这些量子单元上的观点、价值观和态度。

So we actually go collect a lot of these data. Um the kind of data that collection that we might actually do is we care deeply about not just understanding the end outcome or end product and or end message that people care about, but we try to understand the quantum units. So for instance, a So obviously coming from the more visual arts background, what artists would care about is the color scheme we care deeply about. Uh the texture we care deeply about. The overall form and style, so are we minimalist or we formalist or we maximalist? These kind of stylistic choices we care a lot about. So what we first try to understand in the ground truth data is people's views and values and attitudes across these quantum units.

但你们甚至在哪里找到人们对于比如一幅画的感觉的真实数据？

But where do you even find ground truth data of how people feel about say a painting?

所以我们实际上会联系真实的人，在真实的人群中进行研究、调查。所以在任何一周，我们都在与数万甚至数十万真实的人交谈。所以我们每周都了解人们喜欢的颜色类型。这甚至与喜欢什么产品分开，只是在没有上下文的情况下，你喜欢什么颜色？你喜欢什么样的纹理？然后我们会将其渗透到真实情境中。想象你有这种产品。想象你有这种艺术品。你喜欢这个的什么？所以我们实际上会提升到那个水平。你知道，

So we would actually reach out to real people and run studies, run surveys across a real group of humans. So on any given week, we are talking to tens of thousands, if not hundreds of thousands, of real people. So we have a understanding at any given week of the kind of colors that people like. That is even separate from like what product they like, but just in the absence of the context, what colors do you like? What kind of texture are you in for? And then we would percolate that up to real for context. Imagine you had this kind of product. Imagine you have this kind of art pieces. What about this do you like? So we would actually raise it up to that level. You know,

那么，你有没有学到任何关于人性的东西让你感到惊讶？

So, have you learned anything about human nature that surprised you?

这让我惊讶。呃，我认为总是存在这种非常有趣的张力，即人们的态度和行为有多么多样。在某种程度上，它既非常多样，又在某些方面非常可预测。这始终是我一直感受到的一种有趣的张力。如果你与人交谈，他们身上有太多让你惊讶的东西。他们拥有的生活经历，他们拥有的故事。很难理解。同时，一旦你深入了解这个人，你开始理解有一种基本本性往往非常一致，你就开始理解这些人可能如何行为。呃，我最初实际上获得一些初始数据收集灵感的方式是，如果你有家庭成员，他们很了解你。例如，我最亲密的朋友和家人，我的父母，实际上会知道什么让我兴奋，什么让我恼火。呃，拥有这些理解是困难的，但一旦你达到那里，它非常强大。

That surprised me. Um I think there's always this very interesting tension, which is how diverse is people's attitudes and behaviors. And in a way, it's both very diverse and also very predictable in some ways. That always continues to be a bit of a sort of interesting tension that I always feel. Where if you talk to people, there's so many things about them that surprise you. The lived experiences they had, the stories they have. It's very hard to understand. At the same time, once you understand this person deeply and you start to understand there's a fundamental nature that is that tends to be very consistent, you start to have an understanding of how these people might behave. Um the way I initially actually got the inspiration for some of the initial data collection was if you have your family member, they know you quite well. That is sort of for instance, my closest friends and family, my parents, would actually know like what gets me excited, what ticks me off. Um having those understanding is difficult, but once you get there, it's very powerful.

嗯。你认为你能比家庭成员更好地预测某人会做什么吗？

Mhm. And do you think you can predict what someone's going to do better than a family member?

这是个好问题。在某些情况下，呃，是的。

That's a great question. In some context, uh yes.

在哪些情况下你能做出非常好的预测？

What are those contexts where you can make really great predictions?

显然，真正重要的情境是模型有机会更客观一些，但也有机会更加数据驱动。所以，可能有一些关于我家人的情境我可能没有意识到，或者方式上有点难以模式匹配。模型非常擅长模式匹配。

The context where this actually matters a lot, obviously, is the model that does have an opportunity to be a little bit more objective, but also the model that does have the opportunity to be much more data-driven. So, there might be some context that I might not be aware of about my family or in ways that that's a little bit hard to pattern match. The model is very good at pattern matching.

所以，在那些情况下，模型当然可以超越。

So, in those cases, the model can certainly go beyond.

模型可能无法超越的地方，显然是我们没有与这些人深入接触的经验，因为我们花了大约几个小时与人相处来了解他们。这就是有时差距所在。

The places where the model might not go beyond, obviously, is we do not have the depth of experience we had with these individuals because we spent about a couple of hours with people to understand them. And that's where sometimes the gap is.

嗯。好的，所以你提到了这个，但请告诉我你的公司。比如，你什么时候意识到你的研究可以成为一家公司，你是如何开始创业的？

Mhm. All right, so you've alluded to this, but tell me about your company. Like, when did you realize that your research could be a company, and how did you go about starting it?

所以，当我们最初做性别依赖论文并创建整个游戏小镇时，我的许多同事显然采用了那个想法和架构来创建自动化工具，如编码、浏览等。但是，我对模拟感兴趣的原因之一，除了我个人对该领域的迷恋之外，实际上是我们收到了很多入站兴趣。也许因为我们在斯坦福的实验室，但它被一些最大的财富 50 100 强公司的 CEO 和董事会成员注意到了。所以，有些人注意到了，产生了浓厚兴趣，有些人还在斯坦福资助了我们。所以，我们知道如果这项技术真的能够交付，那么可能会有潜在买家感兴趣。

So, when we originally did the gender dependence paper and created this whole game town, many of my colleagues, obviously, took that idea and architecture to create automation tools like coding, browsing, and so forth. But, one of the reasons why I got interested in simulation, in addition to my personal fascination with the with the field, actually was we got a lot of inbound interest. Perhaps because we were at a lab at Stanford, but it got noticed by some of the largest Fortune 50 100 uh companies CEOs and board members. So, some of them noticed, took deep interest, some of them also funded us at Stanford. So, we knew that if this technology could actually deliver, that there would be interest from people who might be our potential buyers.

嗯。

Mhm.

所以，就是这样开始的。当我们发表 2024 年的论文时，我们觉得这些模型的性能已经到位，我有时描述的方式是，感觉很像我们处于 GPT-4 时代，它已经可以产生直接的市场影响。那时我们意识到，嘿，这是值得追求的事情。呃，我一直好奇但深切关注的是，我喜欢未实现的愿景，即愿景的影响真正受限于我们自己的执行，而不是想法或技术的基本原理，感觉非常像那样。我们可以看到这是今天的准确性，这是模拟工具的代表性和力量。但这项技术将继续得到大量研究。在正确的资源和人员下，它每个月都会变得越来越好。这让我们非常兴奋。所以，那时我们决定创办公司。呃，公司本身很年轻，所以我去年六月离开了斯坦福，但我们知道这会引起很多兴奋。

So, that's how it started. And when we published the 2024 paper, we felt like the performance of these models are there, and the way I sometimes describe it is it feels a lot like we're in the GPT-4 era, where it is there to make immediate market impact. And that's when we realized that, hey, this is something that is worth pursuing. Um and the thing that I am always curious about, but that I care deeply about, is I like visions that are unkept, that um impact of the vision is truly capped by our own execution, but not the fundamentals of the idea or technology, and it felt like very much that. Where we could see that this is the accuracies, this is the representativeness and power of simulation tool today. But this is technology that will continue to get a lot of research. It will continue to get better and better with the right resourcing and people every single month. And that got us really excited. So, that's when we decided to start the company. Uh the company itself is young, so I left Stanford June of last year, but we knew that this is something that would get a lot of excitement.

你们成长得非常快。你想谈谈这个吗？

And you've grown really fast. Do you want to talk about that?

是的。所以，现在，呃，团队成长得相当快，嗯，公司周围的愿景和雄心也成长得相当快。所以，联合创始人是我自己、Michael Bernstein、Percy Liang 和 Linyee Allen。Michael 和 Percy 实际上都是斯坦福大学的教授。呃，他们实际上都是我的博士导师，我不知怎么说服了他们加入我的旅程。

Yeah. So, right now, uh the team has grown quite fast, um and also the the vision and the ambition around the company has grown quite fast. So, the co-founders are myself, Michael Bernstein, Percy Liang, and Linyee Allen. Michael and Percy are actually both uh professors at Stanford. Uh they were both actually my doctoral advisors that I somehow convinced to uh to join me in this journey.

Michael 是以人为中心的人工智能领域的领军人物之一，他甚至还共同创建了 ImageNet，所以他是我们今天所见的人工智能革命的早期贡献者。Percy 是当今顶尖的人工智能学者之一。他创造了“基础模型”这个术语，并引领了许多被引入该领域、真正影响人工智能领域的架构。而 Linyee 是在过去三四年里培育出一些增长最快的人工智能原生公司的人。基本上，故事是这样的：我知道我自己、Michael 和 Percy 都是研究人员，但我们知道市场将出现在企业级，我想要一个能与之合作、基本上在企业级发展业务的同事和伙伴，那个人最终就是 Linyee，我有时称她为自然之力。这就是初始创始团队如何走到一起的。

Michael was one of the leaders in human-centered AI, and he actually even co-created ImageNet, so he has been an early contributor to the AI revolution that we see today. Percy is one of the leading AI scholars today. He's the guy who coined the term foundation model, and led to a lot of the architectures that got introduced in the field that really impacted the field of AI. And Linyee is somebody who grew some of the fastest AI-native companies in the past 3 to 4 years. Basically, the story here was I knew that myself, Michael, and Percy are all researchers, but we knew that the market was going to be in enterprise, and I wanted a colleague and partner that I could work with to basically grow the business in enterprise, and that ended up becoming Linyee, who is somebody who I sometimes call a force of nature. So that's how the initial founding team came together.

团队现在增长很快，但截至本周我们大约有 75 到 80 人。但这是一个非常有趣的组合。如今在人工智能领域，有人工智能新实验室和人工智能产品公司；同样，我们代表了一种非常奇特且独特的混合体。公司大约 20% 的人实际上是我自己的实验室伙伴和我的学术兄弟姐妹或亲戚。所以他们中的许多人实际上只是我在 Percy 和 Michael 实验室的伙伴，或者是我学术祖父母的学生。所以我们最近有一位 Eva 团队的成员从伯克利的 Michael Jordan 实验室加入。Michael Jordan 曾是 Percy 的导师。所以一个真正的研究团队正在围绕类似的方向形成。同时，我们确实雇佣了非常有意义的 go-to-market 以及前沿部署的动势。我坚信，在当今人工智能公司建设的时代，前沿部署的动势极其重要。

The team right now is growing fast, but as of this week we're about 75-80 people. But it's a really interesting mix of people. Today in the AI landscape, there are AI neolabs and AI product companies; similarly, we represent something that's a very strange and unique hybrid of both. About 20% of this company is actually made up of my own labmates and my academic siblings or relatives. So many of them are actually just my labmates from Percy and Michael's lab or from students of my academic grandparents. So we recently had a member of Eva's team join from Michael Jordan's lab at Berkeley. Michael Jordan used to be Percy's advisor. So there's a real research team that is getting formed around similarly. At the same time, we do employ a very meaningful go-to-market but also forward deployed motion. I am a huge believer that in this day and age of AI company building, the forward deployed motion is incredibly important.

这就是把你的人放进客户内部，与客户极其紧密地合作，理解他们的需求，甚至构建更针对这些客户的用例和体验，真正建立那种关系。

This is putting your people inside of customers, working extremely closely with customers, understanding their needs, and even constructing use cases and experiences that's a little more tailored to these customers and really building that relationship.

这是我们深切关心的事情，所以我们以非常有意义的方式定位在前沿部署上。然后它与一个惊人的产品和工程团队配对。这个团队显然在持续增长，我们显然在寻找优秀的人才，但这是一个非常有才华的团队。

That is something that we care deeply about, so we're very meaningfully positioned in a forward deployed way. And then it's paired with an amazing group of product and engineering team. That obviously is continued to grow and we're obviously looking for amazing talent, but it's a very talented team.

我的意思是，现在这种情况显然很多，我们在播客上有很多处于你这种情况的人，但我总是好奇从研究员转变为一家估值数十亿美元的大规模公司的 CEO 是什么体验。对你来说怎么样？

I mean this is obviously happening a lot now and we've had a lot of folks on the podcast in your shoes, but I'm always curious what the experience is like going from a researcher to CEO of a company at significant scale valued at billions of dollars. How has that been for you?

所以，这很有趣。我一开始觉得有趣的部分是，这绝对是一种不同的技能组合。运营和管理一家公司与管理一个小型研究团队不同。而且，你为客户提供价值的方式也与为科学界提供价值的方式不同。对于你的客户，需要有非常清晰的投资回报率。你需要能够传达这一点，并且需要伴随很大的确定性。而在研究中，你真正呈现的是愿景和初始原型。但我也觉得兴奋且有趣的部分是，这两者之间有很多相似之处。归根结底，关键在于手艺。归根结底，我们深切关心的是愿景是什么？论点是什么？你能让别人对愿景感到兴奋吗？带他们踏上你的旅程，作为 CEO 你显然会这样做，你提出愿景，你让团队成员加入旅程，你让客户加入旅程。但作为一名研究员，写论文的真正原因是让你的社区对愿景感到兴奋，从而加入你的旅程，甚至作为一个研究团队，你直接招募成员。所以，从那个角度，我实际上以一些奇怪的方式看到了比我进入这份工作时预期更多的相似性。所以，这是一个有趣的转变。

So, it's been interesting. The part that I found to be interesting at first is it's definitely a different skill set. To run and manage a company is different than managing a small research team. And also, the way you deliver value to your customers is also different than the way you deliver value to a scientific community. For your customers, there needs to be very clear ROI. You need to be able to communicate that and it needs to come with a lot of certainty. Whereas in research, what you really present is a vision and an initial prototype. But the part that I also found to be exciting, but also interesting, is there's a lot of similarity between these two. Where at the end of the day it is about craft. At the end of the day, what we care deeply about is what is the vision? What is the thesis? Can you get other people excited by the vision? Bring them to your journey and you do this obviously as a CEO, you present a vision, you bring your own team members to join the journey, you bring your customers to the journey. But also, as a researcher, really the reason to write papers is to get your community excited by the vision that they would come join you on the journey and even as a research team, you recruit your members directly. So, in that way, I've actually seen in some strange ways more similarity than I would have expected coming into this job. So, that's been an interesting transition.

最大的惊喜是什么？

What's been the biggest surprise?

最大的惊喜：最近我从我们的一位投资者那里听说，Simile 作为一家公司以及在那里工作的人，他们身上有某种 Simile 特质。我们雇佣的人和招募的队友，你能看出他们符合这种文化。让我惊讶的是，即使在这个规模上，我们作为一个团队在允许什么样的人进入公司方面仍然极其有主见。其他一些人，社区的其他成员，可以来到这家公司，和我们任何人交谈，并立即感觉到：“哦，你当然在 Simile 工作。”

The biggest surprise: most recently I heard from one of our investors that Simile as a company and the people who work there, there's something Simile about them. And that the people we employ and the teammates we recruit, you can sort of tell that they are like they fit this culture. The part that surprised me is even at this scale, we as a team have stayed extremely opinionated in terms of what kind of people we let into the company. That some other people, other members of the community, can come into this company and talk to anyone of us and can immediately feel like, "Oh, of course you work at Simile."

嗯。

Mhm.

你说话和举止就像属于这个团队的人。我觉得这真的既有趣又在很多方面令人兴奋。

You talk and you behave like somebody who would belong in the team. I thought that was really both interesting and in many ways exciting.

也许是你的一种反映。

Maybe a reflection of you.

我确实认为领导层和领导层所呈现的价值观，尤其是当他们团结一致时，会影响团队的运作方式和团队成长的方式。我们在很多方面能够发出非常清晰的信息，表明我们重视什么，这既是一个惊喜，也是我们一直非常注意的事情。

I do think the leadership and the values the leadership present, especially when they are united, influence the way the team operates and the way team grows. And the way that we were in many ways able to send a very clear message of what we value has been both a surprise but also it's been something that we've been very much mindful of.

在像 Simile 这样的地方，你会有什么价值观，或者我会体验到什么令人惊讶的价值观？就像撇开那种推销说辞。我确信他们很聪明，我确信他们很有能力，我确信他们能自我驱动。但那里有什么令人惊讶的东西？

What's the value that you would have or that I would experience in a place like Simile that would be surprising? Like setting aside the sort of sales pitch. I'm sure they're smart, I'm sure they're capable, I'm sure they're self-starters. But like what's something surprising there?

所以，一是他们深切关心。所以他们显然不仅仅是为了工作而来，而是因为他们深切关心他们在世界上生产的东西以及他们将产生的影响。但同时，我经常告诉人们几件事，我关心的事情，是深切关心的延伸，通常这些人是成功的共同分母。他们处于职业生涯的不同阶段，但对于那些成功的共同分母，他们往往是每个时代他们团队或项目成功的原因。这些人是那种非常擅长重塑自己的人，他们承担极高的主人翁意识，基本上进来说：“不管世界会怎样，我会亲自确保这件事成功。”我们深切关心这种程度的责任感。另一件我认为非常 Simile 的事情是我们喜欢拥有相互矛盾的超能力的人。例如，我有时和我的联合创始人 Michael 谈论这个。

So one is they care deeply. So then they're not just here for the job obviously, but they're here because they care deeply about the things they produce in the world and the impact they're going to have in the world. But at the same time, and a couple of things I always tell people, the things I care about, is an extension of caring deeply is often times these are people for the common denominators of success. That they I have different stages in their careers, but for those who are common denominators of success, they were often the reason why in each era they were the reasons why their team or their project was successful. These are the kind of people who are also incredibly good at reinventing themselves, and who take extremely high ownership and basically come in and say, "Doesn't matter how the world is going to go. I will personally make sure that this is going to be successful." That degree of ownership we care deeply about. Another thing that I think is very Samueleque is we love people who have superpowers that contradict each other. So for instance, I sometimes talk about this with Michael, one of my co-founders.

我早在 2018 年就知道这是我想合作的人，原因在于：作为一名科学家，他是我所在领域有史以来最顶尖的人之一。他极具分析能力，掌握所有正确的技术和方法，是一位极其成熟的科学家。他也是社区里的资深成员。所以他显然具备所有工具，但与此同时，他内心又是个孩子。

The reason why I knew that this was the person I wanted to work with all the way back in 2018 was as a scientist, he is among the very best that my field has ever seen. Extremely analytical. He knows all the right techniques, all the right methods to be an amazingly mature scientist. And he's a senior member of the community as well. So he clearly has all the tool set, but at the same time, inside he's a child.

当我向他提出这个想法——嘿，也许我们可以做一个游戏。我不知道为什么这会是个好主意，但也许这会是件引人入胜的事。当我许多同事感到困惑时，其中许多也是我至今最尊敬的同事，Michael 的反应是：“我也不知道为什么这是个好主意，但这很吸引人。这听起来很有趣。我不知道为什么，但我内心有种东西驱使我，我们必须做这个。”

When I brought him this idea that hey, maybe we can create a game. And I don't know why that's going to be a good idea, but maybe this is something that would be compelling. When many of my colleagues were feeling confused, and many of those colleagues are also the colleagues I respect the most even today, Michael, his reaction was, "I also don't know why that's a good idea, but this is compelling. This sounds fun. I don't know why, but something in me compels me that we have to do this."

拥有那种张力、那种矛盾，因为通常随着你作为科学家越来越成熟，你会失去那种孩子般的好奇。而如果你是个孩子，你显然还没有掌握所有正确的技术。但这两样东西在同一个人身上共存，就为这个人创造了独一无二的强大机会。

Having that tension, that contradiction, because oftentimes as you get more mature as a scientist, you lose that childlike wonder. And if you're a child, you obviously do not yet have all the right techniques. But having those two things coexist in one person creates an amazing opportunity for this person to be uniquely powerful.

所以我们同样深切地关心它。

So that we care deeply about it similarly.

你觉得你也体现了这一点吗？从油画转向计算机科学或人工智能？

Do you feel like you represent that as well, kind of going from oil painting into computer science or artificial intelligence?

从很多方面来说，我最共鸣的部分，当我描述自己时，这很有趣。显然，我既在科学领域待过，也在艺术和社会科学领域待过。从很多方面来说，跨这些学科，我实际上比其他人看到更多相似性。我有点把所有这些都看作是关于创造的学科。

In many ways, the part that I do resonate the most as I describe sort of myself, it is interesting. Obviously, I've been both in scientific fields and I've also been in the art and social sciences. In many ways, across those disciplines, I actually see more similarity across those disciplines than maybe other people do. I sort of see all of those as disciplines that are about creation.

顺便说一句，我同意。是的。

I agree, by the way. Yeah.

所以从那个角度，我觉得就我做的任务而言，我其实相当边缘。然而，我有时喜欢代表自己的那部分，这也同样体现在我所有的领导层中，是我喜欢在日常运营中保持极端的紧迫感。我有时描述自己和 Lanie 为——我通常用她作为真正体现这一点的例子——某种程度的偏执。除非我们做尽每一件可能的事，否则我们就不会成功。

So in that way, I think I've been actually fairly marginal in terms of like the task I've been doing. However, the part that I sometimes the thing that I like to represent myself, and this is also represented across all of my leadership at similarly, is I like to be on day-to-day operation extremely have extreme sense of urgency. And I sometimes describe myself and also Lanie as someone I usually use as an example of somebody who really embodies this. Some degree of paranoia. That unless we're doing every single thing possible, we're just not going to succeed.

嗯。

Mhm.

死亡迫在眉睫。这可能会崩溃。与此同时，我自己、Lanie、Simile 的其他领导层，我确实认为让我们强大的是我们也非常虔诚。我们虔诚地相信，这项技术如果呈现得当、实施得当，将在我们的社会中发挥极其强大、极其积极的作用。而且我们也认为我们会成功。我们看不到一个我们不会成功的未来。我们虔诚地相信，无论过程如何，我们都会以强大的姿态走出来，并做出我们想要的对世界的影响和贡献。

That the death is imminent. That this could crumble. At the same time, myself, Lanie, the rest of the leadership at Simile, I do think what makes us powerful is we are also very religious. We are religious in our belief that this technology when presented right, when implemented right, will be an amazingly powerful, an amazingly positive function in our society. And also we think we're succeed. We don't see a future in which where we won't. We're religious that no matter how this goes we'll come out in a way where we're strong and we have made the impact and contribution to the world that we wanted to.

同时保持这两者，我认为是一个独特的挑战。但我认为这是我非常倾向于的一点，Lanie、Michael 和 Percy 也是如此。我也确实认为需要内心有点破碎的人。

Maintaining those two at the same time I think it's a unique challenge. But I think this is one thing where I've been very much prone to and so is Lanie and so is Michael and Percy. And I also do think need somebody who's a little bit broken inside.

你有没有从自己公司运营的角度使用或思考你的技术？我想到的一件事是，我一个朋友告诉我，在会议前他会描述谁在会议中、主题是什么，然后他会让——我想是 GPT——模拟会发生什么。

Do you use or think about your technology at all in terms of your own company's operations? One thing that comes to mind for me is I had a friend tell me that before meetings he would describe who's in the meeting and what the topic was and he would ask I think it was GPT to simulate what was going to happen.

然后如果会议结果和模拟预测的一样，他就会认为会议是浪费时间，因为没学到新东西。那里没有熵。

And then if the meeting turned out the same way as the simulation predicted, he'd view the meeting as a waste of time because nothing new was learned. There's no entropy there.

这是个好问题。所以在重要会议之前，我确实会尝试用模拟来运行它们。事实上，像我们这样的对话，尤其因为你有非常大的在线存在，很容易模拟。所以是的，你可以打赌我在来之前运行了一些模拟，以了解你可能对问什么样的问题感兴趣。

So it's a good question. So before important meetings, I actually do try to run them in simulation. In fact, conversations like ours, especially because you have a very large online presence very easy to simulate. So yeah, you bet that I did run some simulation before coming in to get a sense of what kind of questions you might be interested in asking.

模拟怎么样？

How was the simulation?

你知道，它……

You know, it's it's

我希望我是不同的。我有点希望我是不同的。

I hope that I'm different. I'm kind of hoping that I'm different though.

嗯，我的意思是，显然有细微差别和不同，但你确实发布了很多采访。

Well, I mean, there are all obviously nuances and differences, but you you did put out a lot of interviews out there.

有趣。

Interesting.

所以，我们在许多模拟中做错了一些事。事实上，内部我们有个叫 siblings 的东西。Siblings 基本上是公司真实成员的智能体或智能体式模拟。这最初其实是我团队一个成员在公司黑客松项目中创建的，因为我们在 Slack 上互相聊得那么多。如果我们拿这个来创建模拟呢？如果我们用那个作为数据来创建公司每个成员的模拟呢？你实际上可以 @ 你的名字或你的生成式 Joon，基本上问那个生成式 Joon 任何你想问的问题。所以，你可以直接问：“嘿，Joon 晚饭想吃什么？”我们有时也这样决定公司晚餐吃什么。但我们也可以问这样的问题：“嘿，Joon 喜欢——你觉得他会批准这个预算吗？”

So, we do something wrong in many of these simulations. In fact, internally, we have this thing called siblings. Siblings basically are agents or agentic simulations of real members of the company. This is actually originally something that one of my team members created as a hackathon project at the company where we talk so much to each other on Slack. What if we take that and create a simulation? What if we use that as our data to create simulations of every member of the company? And you can actually do @ your name or your generative Joon and basically ask that generative Joon any questions you might want to ask. So, you could literally ask, "Hey, what does Joon want to eat for dinner?" We actually sometimes decide what we're going to eat for our company dinner this way, too. But we can also ask questions like, "Hey, does Joon like Do you think he'll approve this budget?"

所以，那个实际上使用量很大。

So, that has actually gotten a lot of usage.

真的吗？嗯，现在我的智能体不能做批准。它们仍然必须来找我，但团队仍然可以问。

Really? Well, now my agent cannot make approvals. They still have to come to me, but the team can still ask.

有趣。我的意思是，我不想往玄乎的方向走，但这会让你思考自由意志吗？以及你真正在选择做事情的程度有多少？

Interesting. I mean, I hate to go in a woo direction, but does that make you think about free will at all and and how much you're really choosing to do things?

是的，这很有趣。我一直觉得那些有点脱节，因为对我来说，自由意志一直更多是关于你承担责任的能力。它不仅仅是关于我会采取什么行动，但对我来说，它更多是关于我对那个行动有所有权吗？如果我采取这个行动并产生某些后果，我会是那个承担责任的人吗？我有权承担责任吗？我认为最终对我来说，至少在情感上，自由意志更多是关于这个。

Yeah, it's interesting. I always saw those to be a little bit disjointed in that for me, free will always has been more about your ability to take responsibility. That it's not merely about what action will I take, but it's for me, it's been more about do I have ownership over that action? That if I take this action and it has certain consequences, am I going to be the one who will take the responsibility and do I have the right to take the responsibility? I think ultimately that to me was more emotionally at least what free will was about.

嗯。

Mhm.

所以从那个角度，我仍然非常相信自由意志。而且我认为能够为自己的行为承担责任，实际上是人类的一大特权。

So in that way I still believe in free will very much. And I think it's actually a great privilege for humans to be able to take responsibility for their actions.

有趣。所以你相信自由意志是因为它可能是个有用的幻觉，还是作为科学家相信它确实是真的？

Interesting. So do you believe in free will because maybe it's a useful illusion versus as a scientist believing that it's actually true?

我认为自由意志是存在的。

I think free will is there.

有趣。好吧。那么，这可能是结束的好地方。

Interesting. All right. Well, that might be a good place to stop.

不过，我其实真的想问你：那个模拟的我有没有想出什么我没问过你的好问题？

Although, I do actually want to ask you: did the simulation of me come up with any good questions that I haven't asked you?

有的，我觉得我们其实覆盖得挺全面的。所以那是一次很有意思的对话。

Yeah, I think we actually had pretty good coverage. So that was a lot of fun conversation.

好的。那么，谢谢你，Joon。非常感谢。

All right. Well, thanks, Joon. Really appreciate it.

彼此彼此。

Likewise.

太棒了。非常感谢大家收听本期 Greater Descent。请继续关注后续节目。

Awesome. Thanks so much for listening to this episode of Greater Descent. Please stay tuned for future episodes.