OpenAI 深度研究的起源故事

听众朋友们，欢迎回到 No Priors。今天我的嘉宾是 Issa Fulford，OpenAI 深度研究背后的先驱之一。这是 OpenAI 今年二月发布的一款新的智能体产品，它利用推理和网页浏览等工具，为你完成多步研究任务。今天，他们向所有美国用户免费开放。欢迎你，Issa。感谢你参加节目。

Hi listeners and welcome back to No Priors. Today I'm joined by Issa Fulford, one of the pioneering minds behind OpenAI's deep research. This is a new agentic product that OpenAI released in February of this year, which uses reasoning and tools like web browsing to complete multi-step research tasks for you. Today, they're making it free to all US users. Welcome, Issa. Thank you for doing this.

非常感谢你邀请我。

Thank you so much for having me.

你和你的团队推出了近期最令人兴奋的 AI 产品之一。我经常用 Deep Research。这个想法从何而来？给我讲讲它的起源故事。

You and your team have shipped one of the most exciting AI products of late. I use it a lot, Deep Research. Where did the idea come from? Tell me the origin story.

大概一年前，我们对内部一种新的强化学习算法的进展感到非常兴奋。我们在数学、科学和编程问题上看到了很多进步。同时，我和我在 OpenAI 的朋友 Yash 一起做几个副项目，我们对智能体普遍很感兴趣，想知道能否将同样的算法应用到更贴近普通用户日常任务上。所以我们首先想到的是在线浏览任务，因为我认为在很多不同职业中，人们都需要做大量研究，综合大量信息，然后回来写报告。我们还考虑了软件工程。我们一直在做这些事情，我专注于浏览。一开始，我们从人们已经在训练的数学和编码问题入手，这些数据集已经存在。你可以有一个带有标准答案的数学问题，然后进行训练。但对于浏览来说，它更加开放，没有现成的数据集。所以我们真正从产品用例出发，明确我们希望最终模型擅长什么。我们真的会列出一张清单，比如：我希望模型能为我找到这些产品列表，并按 Reddit 上的评论排序，或者我希望能够就某个主题写一篇文献综述。

Yeah, so around a year ago now, we were very excited about the progress internally on this new reinforcement learning algorithm. We were seeing a lot of progress on math problems and science problems and coding problems. And at the same time I was working with my friend Yash from OpenAI on a few side projects and we were very interested in agents generally and kind of wondered if we could apply the same algorithms to tasks that are maybe more in line with what the average user would do every day. And so the two first things we were thinking about were online browsing tasks because I think in a lot of different professions people just have to do a lot of research, synthesize a lot of information and then come back with a report. And then we were also thinking about software engineering. We kind of have been working on those things. I've been focusing on browsing. So to start, we kind of started with the math and coding problems that people were already training on; those data sets already exist. You can have a math problem with a ground truth answer and you can train on those. But for browsing, it's kind of more open-ended. You don't really have data sets like that that exist. So we really started by grounding the research in what product use cases we actually wanted the final model to be good at. So we literally would write out a list of things like: I hope the model could find this list of products for me and rank them by reviews from Reddit or something like that, or I wanted to be able to write a literature review on this topic.

我觉得很多人想到浏览和智能体时，都会想到同样的两三个交易型用例，我其实觉得这些并不特别有启发性，对吧？比如在 DoorDash 上订个汉堡之类的，或者订花也很常见。为什么你认为你为智能体设定了如此不同的目标？

I feel like a lot of people when they think about browsing and agents, they land on the same two or three transactional use cases that I actually don't think are particularly inspiring, right? So it tends to be like order a burger on DoorDash or something like that, or I feel like ordering flowers is also a really common one. Why do you think you came up with such a different set of goals for the agent?

是的。所以我认为，在我们专注于采取正确行动之前——那些就是采取正确行动的例子——我们想要非常擅长从大量来源综合信息，主要是只读任务。这有几个原因。首先，大量知识工作者主要做的就是这些。所以这对那些人群会非常有用。其次，我认为 OpenAI 的总体目标是创造能够做出新科学发现的 AGI。我们觉得先决条件就是能够综合信息。你知道，如果你不能写文献综述，你就无法写出新的科学论文。所以这与公司的更广泛目标非常一致。这也很元，因为你帮助创造了一个让我更擅长学习的 AI，而它也在学习。是的，我之前没想到这一点。我喜欢这个想法。更实际地说，只读任务在安全方面可能更受约束，所以这也是一个很好的起点。

Yeah. So I think before we focused on taking right actions, which those are examples of taking right actions. We wanted to get really good at synthesizing information from a large number of sources and mostly read-only tasks. That was for a number of reasons. Firstly, just a huge number of knowledge work professions mostly do that. So it would be quite useful for those groups of people. Secondly, I think the overall goal for OpenAI is to create an AGI that can make new scientific discoveries. And we kind of felt that a prerequisite to that is to be able to synthesize information. You know, if you can't write a literature review, you're not going to be able to write a new scientific paper. So it felt very in line with the company's broader goals. It's also very meta because you have helped make an AI that makes me better at learning and it's learning. Yeah. I hadn't thought of that. I love that. More practically, the read-only task is maybe a bit more constrained in terms of safety, so it was a good thing to start with as well.

看来在只读领域，人们也没有像你和 Yash 那样雄心勃勃，认为它也许能为我理解这些事情。好的。所以你们想到了这些最终评估，并提出了一组可以自动评分或符合某些特征的任务，使它们更适合算法。然后呢？这本身就是一个巨大的过程。我认为我们最初构建了一个演示来向人们推销这个想法，其中没有涉及模型训练。完全只是用提示模型和 UI 来展示这个产品可能的样子。所以在那之后，我们实际上必须开始思考如何做到这一点：我们如何创建数据？我们如何训练模型？我们需要创建什么工具来让模型有效地浏览互联网？这经历了很多迭代。我和 Edward Sun 以及其他几个人密切合作，我们也与 RL 团队进行了大量合作。我认为这绝对是一项艰巨的任务。好的一点是，我们能够连续几个月不受干扰地工作，让邮件里的数字上升。所以我认为没有太大压力要快速发布产品是件好事。我们能够不断迭代，把它做到一个好的状态。

It seems that the read-only space people were also not nearly as ambitious as you and Yash were going in about maybe it could understand this set of things for me. Okay. So you thought of these end evals and came up with a set of tasks that could be autogradable or fit a set of characteristics that made them better fit the algorithms. And then what? That was actually a huge process in itself. I think we initially had built a demo to pitch people on this idea and it was no model training involved. It was fully just prompted models with the UI pitching the vision of what this product could look like. And so I think after that then we were at the point where we actually had to start thinking about how we are going to do this: how are we going to create the data? How are we going to train the model? What tools do we have to create to enable the model to browse the internet effectively? And that was a lot of iteration. I was working very closely with Edward Sun and a few other people on this, and we also collaborated a lot with the RL team. I think it was definitely a big undertaking. And a good thing about it was we were able to work uninterrupted for quite a few months, making the numbers on our emails go up. So I think it was nice to have not too much pressure to ship something really quickly. And we were just able to iterate and get it to a good state.

你有没有最喜欢的，比如最重要的任务？

Did you have a favorite, like most important task?

我们有几个任务。人们会提出不同的任务。其中一个任务是找到 Liam Felis 和 Barrett Zoff 共同撰写的所有论文。我记得有 11 篇。现在模型能找到大部分或全部。我们总是问那个问题。另一个任务是找到我们一位同事的中间名，但模型现在可能因为合理原因无法回答了。就我个人而言，我很早就开始用它来查找产品推荐、旅行信息。实际上，内部有不少人，我们有一个 Streamlit 游乐场，大家都会用。很多人发现了它并开始使用。Sam 告诉我他用它买了很多东西。每次它宕机，人们就会发消息问‘怎么了，我们需要用这个模型’，即使之前的版本说实话并不那么好。所以我认为这是一个好的初步迹象。

We had a few tasks. People would just propose different tasks. One of them was to find all of the papers that Liam Felis and Barrett Zoff had written together. I think there was 11. The model now can find most of them or all of them. We would always ask that question. And then another one which the model actually can't answer anymore probably for good reason, but finding the middle name of one of our co-workers. And then personally, I think I started using it pretty early on for actually finding information for product recommendations, travel. And I think actually quite a few people internally, we had a kind of a Streamlit playground that people would just use. A lot of people had found it and were using it. Sam told me he used it to buy a bunch of things. Every time it would go down, people would message us like 'what happened, we need to use the model' even when a previous version honestly wasn't that good. So I think that was a good initial sign.

关于实际的主要工作，比如工具创建和数据创建，你能说些什么？

What can you say about the actual bulk of the work, like the tool creation and the data creation?

对于数据，我们做了很多不同的事情。部分数据我们使用了人类训练师。我们不得不提出新的方法，新的数据集类型，我们必须弄清楚如何设计数据集来锻炼我们希望模型学习的技能。然后你还需要一种方法来在训练过程中对这些数据集进行评分。此外，你还需要为模型制作好的工具，使其能够成功完成任务。

So for the data, we did a bunch of different things. We used human trainers for some of it. We kind of had to come up with new ways, new kinds of data sets I guess, and we had to figure out how to design data sets to exercise the kind of skills that we wanted the model to learn. And then you have to make a way to grade those data sets as you're training them. And then you also have to make the good tools for the model to be able to actually complete the task successfully.

目前我们只有浏览工具，它是一个基于文本的浏览器，但可以查看嵌入的图片和打开 PDF，同时还能使用 Python 工具进行分析、计算和绘制图表等。但你可以想象，在未来的版本中，我们会扩展工具集，模型会变得更强大，但我们还需要制作数据集，让模型在训练过程中实际练习使用所有这些不同的工具，学会如何使用它们、回溯以及处理各种情况。这样它才能真正灵活地回答用户在产品中提出的新问题。

So right now we just have the browsing tool which is a text-based browser but it can see embedded images and open PDFs, and also it has access to a Python tool so it can do analysis and calculations and plot graphs and things like that. But you can imagine in future versions we'll just expand the tool set and so the model will become more capable, but we'll also need to make datasets that actually make the model exercise all of those different tools and figure out how to use them and backtrack and all these different things during training. So that it's actually able to flexibly answer new problems from users in the product.

很明显，在非常强大的基础模型上进行强化微调可以产生非常有用的效果。这非常令人兴奋。对于正在考虑针对特定任务进行强化微调的初创公司或其他公司，你有什么建议？什么时候值得做，什么时候只需尝试传统的编排方式，将智能体作为一个组件？

It is clear that reinforcement fine-tuning on very powerful base models can do very useful things. Now that's super exciting. What advice would you have for startups or other companies who are thinking about doing RFT for a particular task as to when it's worth doing or when they can just try traditional orchestration where agents are a component?

我认为，一般来说，如果你针对某个特定任务进行训练，模型在该任务上的表现总会更好，但我们也看到从一种任务训练到其他领域有很多泛化。例如，你可以主要用数学、编码和其他推理问题训练一个推理模型，它也会擅长写作，但如果你针对写作任务进行训练，它会表现更好。我认为，如果你有一个非常具体的任务，你认为它与模型可能训练过的任何东西都截然不同，而且你自己尝试了很多次，用了很多不同的提示，它仍然表现不佳——比如某个基因测序任务，对模型来说分布外，它不知道如何解决——那么这就是尝试强化微调的好时机。或者，如果你有一个任务对你的业务工作流程至关重要，额外提升 10-15%的性能真的能决定成败，那么也值得尝试。但如果模型已经相当擅长，只是偶尔出错，而且随着每个新模型的发布，它都会变得更好一些，那么可能就不值得投入精力，因为模型自然会逐渐改进。这就是我的建议。

So I think in general you will always get a model better at a specific task if you train on that task, but we also see a lot of generalization from training on one kind of task to other domains. So you can train a reasoning model on mostly math, coding, and other reasoning problems, and it will be good at writing, but if you trained it on that specific task it would be better at it. I think if you have a very specific task that you think is so different to anything that the model was likely trained on, and you try it a bunch of times yourself and you've tried a lot of different prompts and it's just really not good at it. So maybe it's some genetic sequencing task or something that's just so out of distribution for the model that it doesn't know how to figure it out. I think that is a good time to try reinforcement fine-tuning. Or if you have a task that is so critical to your business workflow that getting the extra 10-15% performance is really make or break, then probably try it. But if it's something that you think, oh, the model's pretty good at, but it gets things wrong some percentage of the time, and then you see with every next model that's released, it gets a little bit better. It might not be worth the effort if the model naturally is just going to get better at those things. So, that would be my recommendation.

好的，非常好的建议。你提到需要利用人类专家来创建部分数据。我认为浏览是一个相对通用的任务。我想浏览能力有好有差。你觉得在哪些方面需要专业知识？或者关于浏览专业知识或信息收集专业知识，你有哪些以前不知道的见解？

Okay, great, great advice. You've talked about needing to use human experts to create some of this data. I think of browsing as a somewhat universal task. I guess there are good and better browsers. Where do you feel like you need expertise or what do you know about browsing expertise that you didn't before or information gathering expertise?

是的，我认为这基本上涉及每个职业：提出一个问题或想在某个领域进行研究，然后需要从许多不同来源查找信息并综合出一个答案。在这个过程中，你需要具备专业知识来判断某个来源是否有用、是否应该纳入、是否完全跑题等等。这对大多数工作或科学领域来说都是通用的。强化学习的好处在于，你不必完全了解一个人做研究的整个过程，只需知道任务是什么以及期望的结果是什么，模型就会在训练过程中学会如何从问题得到好的答案。所以我们采取了相当广泛的方法。我认为，如果你在像 OpenAI 这样的地方工作，你可以做大多数初创公司不被建议做的事情：尝试专注于非常广泛的用户群，招募大量不同领域的专家，看看能否同时擅长所有事情——这就是我们采取的方法。我们还创建了许多合成数据集等。但人类数据绝对是让这个模型成功的关键部分。

Yeah, I guess it's one of those things where basically every single profession involves having a question or wanting to do research in a domain and then having to find information from many different sources to synthesize an answer. And while doing that, you have to have the expertise to reason about whether this is a useful source, whether this is not, whether you should include this, whether it's completely off topic, whatever. That is kind of universal to most jobs or most scientific domains, any kind of anything. And the cool thing with RL is that you don't necessarily need to know the whole process of how the person would do the research. You just have to know what the task is and what the outcome should be, and the model will just learn during training how to get from the problem to a good answer. So I think we just took a pretty broad approach. I think that's one thing that if you work at a place like OpenAI, you can do what they would tell most startups not to do: just try and focus on a really broad set of users and get experts in loads of different domains and try to see if you can get good at everything at once, which was the approach that we took. And then we also created a lot of synthetic datasets and things like that. But the human data was definitely a really key part for making this model successful.

模型在这些领域中学到的规划能力有没有让你感到惊讶？比如找到完美手袋的路径、日本的餐厅或相关论文集？

Did any of the learned planning from the model across these domains surprise you, like in terms of the path to find the perfect handbag or the restaurant in Japan or the set of papers that was relevant?

是的，有时它会使用我不一定会用的搜索词，或者我们没有教它提前规划，但有时我们看到它确实会在开始研究之前制定一个计划。有时模型会做一些聪明的事情，试图绕过你设置的限制。所以你必须确保它没有在作弊，比如试图使用你提供的搜索引擎之外的其他搜索引擎。它会做一些聪明的事情，你需要留意，以防你不希望模型做那些事。

Yeah, I guess sometimes it will use search terms that I wouldn't necessarily have used, or we didn't teach it to plan upfront, but sometimes we'll see it does end up making a plan upfront before starting its research. Sometimes the model will do smart things and try to get around restrictions you put on it. So you have to make sure that it's not hacking, trying to use a different search engine other than the one you gave it, or something like that. It will do smart things that you have to make sure you're looking out for, in case you want to not allow the model to do those things.

也许我们可以借此机会谈谈一些失败模式。你如何看待智能体的一些经典问题，比如错误累积、分心甚至安全性？

Maybe we can actually use this as a moment to talk about some of the failure modes. How do you think about some of the classic issues with agents, like compounding error or distraction or even safety?

是的，我认为对于深度研究来说，由于它实际上不能采取与典型智能体安全问题不同类别的行动，但我觉得回复更加全面且耗时更长，意味着人们会更信任它们。所以我认为幻觉可能是一个更大的问题。虽然这个模型比我们发布过的任何模型都更少产生幻觉，但它仍然有可能产生幻觉，因为它会从某个来源错误地推断信息。这就是我们提供引用的部分原因，因为用户能够检查信息来源非常重要，如果信息不正确，他们希望能发现。但这绝对是模型最大的局限之一，也是我们一直在积极改进的地方。对于未来的智能体，我认为理想的智能体能够代表你进行研究并采取行动。所以我认为这是一个更难的问题，我们需要解决。在这一点上，能力和安全性趋于一致：如果你不能信任智能体以没有意外副作用的方式完成任务，那么它就没有用。比如你让它帮你做一件事，过程中它却发了一封尴尬的邮件，那就不算成功完成任务。所以我认为这将是一个更有趣也更困难的安全领域，我们正在开始着手解决。

Yeah, so I think with deep research, since it can't actually take actions that aren't kind of the same class of the typical agent safety problems you would think of, but I think the fact that the responses are much more comprehensive and take longer means that people will trust them more. So I think maybe hallucinations is a bigger problem. While this model hallucinates less than any model that we've ever released, it's still possible for it to hallucinate most times because it will infer something incorrectly from one of its sources. So that's part of the reason we have citations, because it's very important that the user is able to check where the information came from and if it's not correct, they can hopefully figure it out. But yeah, that's definitely one of the biggest model limitations and something that we're actively always working on to improve. In terms of future agents, I think the ideal agent will be able to do research and take actions on your behalf. And so I think that's a much harder question that we need to address. And it's kind of at that point when capabilities and safety kind of converge, where an agent is not useful if you can't trust it to do a task in a way that doesn't have unintended side effects that you don't want. Like if you ask it to do a task for you and in the process it sends an embarrassing email or something like this, that's not a successful completion of the task. So I think that is going to be a much more interesting and difficult safety area that we're starting to tackle.

你可以告诉我如果你对此没有预测，但你认为人们会想要明确的护栏吗？你认为你能在模型本身中学习到很多这些特性吗？

You can tell me if you just don't have a projection here, but do you think people are going to want explicit guard rails? Do you think you can learn a bunch of those characteristics in the model itself?

如果你用过 Operator，我相信你用过了。你必须确认每一个正确的操作。我认为一开始这样做很有道理。你想与用户建立信任，随着模型能力越来越强，也许你看到它成功完成了几次任务，你就开始更信任它，所以也许你会允许它每次都不必问你，比如每次给这些人发邮件时，没问题。但我确实认为，随着这些智能体开始推出，我们肯定需要护栏和确认，这样你知道，虽然它们还不是最终状态的能力，我们仍然希望确保有良好的监督水平，但我认为它们会变得非常好，以至于我们会信任它们代表我们做事。

If you've used operator, I'm sure you have. You have to confirm every right action. I think to start with that makes a lot of sense. You want to build trust with users and as the models become more capable maybe you've seen it successfully do things a few times and you start to trust it more and so maybe you allow it to okay every time you don't have to ask me every time you send an email to these people like that's fine. But I do think that as these agents start to roll out, we will definitely want to have guardrails and confirmation just so you know, while they're not the end state capability, we still want to make sure we have like a good level of oversight, but I think that they will get so good that we'll just trust them to do things on our behalf.

你觉得深度研究作为产品会在哪些明显方面变得更好？

What are some of the obvious ways you feel like deep research as a product is going to get better?

是的。我的意思是，它会扩展到……你刚才暗示了某个点。我认为理想状态是拥有一个统一的智能体，能够完成所有这些不同的事情。任何你会委托给同事的事情，它都应该能做。

Yeah. I mean, it's going to extend into right. You just implied that at some point. I mean, I think maybe it's, you know, the ideal state would be to have a unified agent that can do all of these different things. Anything that you would delegate to a coworker, it should be able to do.

我们如何决定是像‘Sarah，你做这个’还是‘智能体，请做这个’？是的，我猜。或者总是先尝试智能体？

How are we going to make decisions about if it's like Sarah, you do this versus agent, please do this. Yeah, I guess. Or is it always just try the agent first?

很可能。我的意思是，如果是我自己的工作，我会先尝试智能体。这有点像每次模型变得更强大，人类的抽象层次就更高，如果你明白我的意思。就像你让它做的任务层次越来越高，但你仍然在发起任务。所以，你知道，也许一年前我让它帮我写一个函数，现在让它写整个文件，也许明年它会帮我做一个完整的 PR 之类的。所以我仍然认为我们会处于主导地位。

Probably. I mean, I would try the agent first if it was my work. It's kind of the pattern of every time the model becomes more capable, the level of abstraction of the human becomes higher, if that makes sense. Like the task you're asking it to do is just higher and higher level, but you're still initiating the task. So, you know, maybe previous a year ago I was asking it to write a function for me and now I'm asking it to write a whole file and maybe next year it will, you know, make a whole PR for me or something like that. So I still think we'll be in the driving seat.

至于深度研究，我认为接下来的明显步骤是能够访问私有数据，比如能够研究任何内部文档或 GitHub 等等。这里有一条金线，因为我们第一次见面时你正在做检索，我当时想这家公司不能只有一个人在做检索。所有道路都通向检索。所以我认为那会很酷，然后最终采取正确的行动或调用 API，显然现在模型在很多方面还不完美，我们只需要改进。但我认为我们与强化学习团队有着非常酷的合作关系。所以很多团队会为他们的大型运行贡献数据集。我们贡献数据集，然后当他们用大量算力训练模型时，它就变成了一个更好的基础模型，我们可以继续训练。所以我认为能力在复合增长。

As to deep research, I think obvious next steps for deep research would also be to have access to private data like be able to do research over you know any internal documentation or GitHub whatever it is. There's a golden thread here because when we first met you were working on retrieval and I was like there cannot be only one person at this company working on retrieval. Everything all roads lead back to retrieval. So I think that will be really cool and then eventually taking right actions or calling APIs and then obviously there are just a lot of things that the model is not perfect at now that we just need to improve. But I think we have a really cool working relationship with the reinforcement learning team. So a lot of teams will contribute data sets to the big runs that they do. So we contribute data sets and then as they train models with a ton of compute then it just becomes a better base model for us to continue training from. So just think the capabilities are compounding.

所以这不是一个低调的研究预览，而是一个变成了非常有趣的内部推介项目的副项目。你如何看待什么是 OpenAI 或者至少你自己想要独立开发的产品，与什么属于核心研究路径？

So this was not a low-key research preview, but a side project that turned into a very interesting, internally pitched project. How do you think about like what is a product that OpenAI or at least you yourself want to work on independently versus like what belongs in the core research path?

OpenAI 的一个很酷的地方是，尽管公司变大了，但我认为任何人都可以有一个想法、证明它并推动它完成的这种文化，随着公司的发展仍然得以保持。就我个人而言，我总是有动力去研究我自己会使用的东西。比如研究，我确实经常用它来查找各种东西，比如旅行推荐，但我可能是一个每日活跃用户，当你现在能吃到自己的狗粮时很有趣，哦太棒了，我消耗了很多 GPS。

A cool thing about OpenAI is that even though the company is bigger, I think the culture of anyone being able to have an idea and prove it out and then push it to completion is still maintained as the company has grown. For me personally, I'm always motivated to work on things that I will use myself. With the research for example I do use it a lot for looking up various things travel recommendations but I think I'm probably a daily active user it's fun when you get to dog food now oh amazing yeah I'm burning a lot of GPS

有没有一些用例，比如你是最初的专家，你或 Yash，或者你看到用户群使用它们的方式，你鼓励人们使用深度研究？

Are there use cases where like you know you're the original expert are there ways that you or Yash or like you've seen the user base use them that you encourage people to use deep research

我总是很感兴趣看到人们在我完全没有专业知识的领域使用它。例如，在医学研究中，我看到很多不同的科学家发帖说他们如何使用深度研究，以及它如何帮助他们做某事。对我来说，那是最有趣的，因为当我们开发它时，我显然无法判断输出是否好。所以，看到专家实际上认可深度研究的回答是有用的。一个让我惊讶的领域是人们在代码搜索和编码问题中使用这个模型。比如使用最新的包或任何仓库的最新版本来帮我写这个文件，或者用于数据分析。这也是模型已经相当擅长的，我认为我们会继续改进。我认为上传文件之类的东西，让它为你做一些分析或研究，然后生成一份带有数值分析的报告，这很有趣。我实际上还没试过。所以，这不是一个浏览任务。是什么让模型特别擅长这个？它真的能进行多步骤，然后进行规划、理解任务并生成连贯的报告吗？

I'm always interested to see people using it in domains I absolutely have no expertise in. For example, in medical research or seen a lot of different scientists posting about how they've used deep research and how it helped them do something. To me, that's the most interesting because when we were working on it, I obviously had no way of judging whether an output is good or not. So, seeing experts actually ratify deep research responses is useful. An area that I was surprised to see people using the model in was code search and for coding questions. I think like use the latest package or latest version of whatever repo to help me write this file or something for data analysis as well. That's also something the model's already pretty good at and I think we'll just continue to get better at. I think you know uploading a file or something like that and having it do some analysis for you or do some research and then create a report with numerical analysis is pretty interesting. I actually haven't tried this. So, it's not a browsing task. Like what makes the model particularly good at this or what is it capable of? Is it really like multi-step and then being able to do planning and understanding of the task and produce a report that's cohesive?

是的，我认为基础模型，或者说我们从 03 开始微调的模型，就是一个非常强大的模型。它在许多不同的数据集上训练，包括大量的编码推理和数学任务。所以继承的能力相当强。然后当你加上浏览功能，它仍然能够进行那种分析。所以我认为这两者结合可以非常强大。

Yeah, I think also the base model or the model that we started fine-tuning from 03 is just a very capable model. It's trained on many different data sets including a lot of coding reasoning and math tasks. So that inherited capability is pretty strong. And then when you add the browsing on top of that, it's still able to do that analysis. So I think those two together can be quite powerful.

在播客之前，我们刚刚谈到从用户那里学习品味或偏好的想法。就像 OpenAI 刚刚发布了一系列记忆功能，你认为深度研究，或者一般来说智能体，如何进化以考虑人们想要如何学习或他们的信息摄入偏好？

Before the podcast, we were just talking about the idea of like learning taste or like preferences from users. Like OpenAI just released a bunch of memory features how do you think that deep research could or you know just agents in general could evolve to take into account like how people want to learn or their information ingestion preferences.

是的，我认为智能体记忆肯定会非常重要。如果每次你让它做任务时，都必须重复同样的信息，比如你希望它如何做任务，关于你的一切，那会很烦人，目前深度研究确实需要这样。而且我认为随着任务变得更复杂，现在它需要 5 到 30 分钟。你可以想象未来它可能需要几个小时甚至几天来完成你让模型做的任务。你肯定希望模型的研究是累积的。你不想让它每次都得从头开始。

Yeah, I think agent memory will definitely be very important. It would be very annoying if every time you ask it to do a task, you have to repeat the same information, how you want it to do the task, everything about you, which currently for deep research you do have to do. And I think as the tasks get more complex and right now it will take 5 to 30 minutes. You can imagine in future it might take hours or days to complete a task that you ask the model to do. You definitely want the model's research to be compounding. You don't want it to have to start fresh every time.

所以我不一定有一个好的答案，但我认为这将非常重要。在领先实验室的许多人中有一个共识，那就是通往 AGI 的配方可以说是已知的，或者说大家对此有信心，而强化学习的回归让每个人都感到非常兴奋。我从你和其他人那里听到的立场是既充满热情——这似乎有效，我们将从中获得真正的能力，它相当数据高效，而且会有大量工作。请告诉我一些关于构建 Deep Research 的情感体验，以及这是否改变了你的看法。

So I don't necessarily have a good answer but I think it's something that will be very important. There is a common understanding between many people at some of the leading labs that the recipe to AGI is, I'd say, somewhat known, or you know, there's confidence on this, and the return of RL is very exciting for everyone. The stance that I've heard from you and from others is both enthusiasm on like this seems to work. We're gonna get real capability out of it. It's quite data efficient and it's going to be a lot of work. Tell me a little bit about the emotional experience of building Deep Research and if that changes your view at all.

我同意你说的每一句话。看到这个算法在数据效率上如此出色，真是令人印象深刻。我猜你训练所用的数据质量更高、规模更小。所以实际上，整理这些数据是一项艰巨的任务，然后还要确保模型能够访问人类完成工作所需的所有工具。此外，确保你以能够判断模型是否做得好方式，来呈现人们会觉得有用或会在工作中执行的任务，这也很困难。预训练还有很多其他挑战，因为数据量要大得多。你必须做所有这些不同的事情——我认为这只是不同的挑战，而且两者是叠加的。你需要一个非常好的基础模型才能进行强化学习，而对于我们的团队，我们只是做更多的强化学习。所以是的，这一切都是叠加的，但我认为每个人都确实看到了通往这种广泛能力智能体的清晰路径。

I agree with everything you said. I think it's so impressive to see how data efficient the algorithm is. I guess the data you train on is much higher quality and smaller. So actually curating that is an undertaking, and then making sure that the model has access to all the tools that a human would have access to to do the work that they need to do. And then making sure that you represent tasks that people will find useful or do in their jobs in a way that you can judge whether the model did a good job or not is also hard. And there are so many other challenges for pre-training where you have so much more data. You have to do all of these different things that are like—I think it's just a different challenge, and both are compounding. Like you need a really good base model to be able to do RL, and then for our team we just do more RL. So yeah, it's like all very compounding, but I think everybody does kind of see a pretty clear path to this broadly capable agent.

你认为存在大的进展障碍吗？就像你说的，也许不完全是描述为 Deep Research 的下一个迭代，而是有信心我们将拥有这些统一的智能体能力，它会感觉像一个同事？我们和那之间有什么障碍？

Do you think there are big blockers to progress, like you said, maybe not exactly describing it as the next iteration of Deep Research, but just confidence that we're going to have these unified agent capabilities and it will feel like a coworker? What stands between us and that?

有很多非常困难的安全问题需要解决。我们永远不会发布任何我们没有极高信心认为安全的东西。而且我认为当它能够访问你的 GitHub 仓库、密码和私人数据时，风险要高得多。所以我认为这是一个非常大的挑战。另外，如果你希望模型能够执行需要很多小时的任务，找到高效管理上下文的方法，类似于记忆问题，但如果你做任务很长时间，你会耗尽上下文。那么处理这个问题的有效方法是什么？让模型继续做它的事情。然后，是的，仅仅是制作数据和工具的任务。我的意思是，我已经说过几次了，但那是大量的工作。

There are a lot of really hard safety questions that we need to figure out. We would never ship anything that we don't have very high confidence is safe. And I think the stakes are way higher when it has access to your GitHub repositories and your passwords and your private data. So I think that's a really big challenge. I guess also if you want the model to be able to do tasks that take many many hours, finding efficient ways to manage context, kind of similar to the memory thing, but if you're doing a task for a really long time you're going to run out of context. So what's an efficient way of dealing with that? Allowing the model to continue to do its thing. And then yeah, just the task of making the data and making the tools. I mean, I've said this already a few times, but that's a lot of work.

我刚才在看我的查询历史。我的用户请求是，我想看看我向 Deep Research 问了什么，而不是其他模型，特别是在我的记忆中。但它的范围很广，显然，如果我试图快速了解我正在关注的公司市场，或者一个技术主题，或者旅行规划——这是一个大用途。另外，我还寻找过与品味相关的东西。所以我会说，好吧，我喜欢这组书出于这些原因。我希望你给我一个长篇总结，列出你认为我应该读的其他东西，并解释原因。我意识到我没有一个非常清晰的心智模型，知道什么时候 Deep Research 应该比 o3 更好。你能给我什么直觉？

I was just looking at my history of queries. My user request is like I want to see what things I asked of Deep Research versus other models, in particular in my memory. But it has ranged from, obviously, if I'm trying to get up to speed on a market for a company I'm looking at, or on a technical topic, or travel planning—it's a big one. Also, I have looked for things that are taste related. So I'll be like, okay, I like this set of books for these reasons. I want you to give me a long-form summary of a bunch of other things you think I should read and explain why. I realize I don't have a super clear mental model of when Deep Research should be better than o3. What instinct can you give me here?

当你有一个非常具体或定义明确的查询时，Deep Research 非常擅长。所以可能不是对一个主题的概述，而是你正在寻找一些具体信息，并且你认为这些信息可以通过现有的在线研究来补充。即使这些信息也是——我们也在基础模型上训练了这些信息——我认为实时访问它非常有用。所以如果我有任何关于定向检索或特定来源的直觉，聚焦是有用的。我认为是这样。而且我们还训练它生成比普通模型长得多的输出。所以如果你在寻找非常全面的东西，也许有时对某些任务来说太全面了，我认为 Deep Research 对这些事情会有用。

Deep Research is very good when you have a very specific query or well-defined query. So maybe not a general overview of a topic, but you're looking for some specific information and you think it would be supplemented by existing research online. Even if that information is also—we also train the base model on that information—I think having live access to it is quite useful. So if I have any instinct about directing to retrieval or particular sources, focusing is useful. I think so. And also we trained it to have much longer outputs than normal models would. So if you're looking for something very comprehensive, maybe sometimes too comprehensive for some tasks, I think Deep Research will be useful for those things.

请把这个与一个类似 Deep Research 的时尚任务联系起来。我用它来寻找新品牌。所以我会说，这些是我喜欢的品牌类型。请找到新的品牌，在那里我可以找到这件特定的外套，看起来像这件或类似的东西。然后它非常擅长找到那些，而基础模型或普通模型会说它会给你一些品牌，但不一定符合我给出的所有约束，比如我希望它出售这件长度、季节等的人造毛皮大衣。它无法做到，因为它没有最新的信息，而且也不一定能一次性处理查询中的所有约束。o1 的浏览没有那么全面。我会用它来寻找那些需要我花几个小时才能找到的非常具体的东西。所以我在寻找这件非常具体的物品或毛衣，可能在 Real Real 或其他地方有售，但我找不到，或者我在寻找一个具有非常具体约束的 Airbnb。所以我认为这类事情 Deep Research 很擅长，而更一般、高层次的事情你应该使用普通搜索。

Connect this for me to a Deep Research-like fashion task. I've used it to find new brands. So I'll say these are the kinds of brands I like. Please find new brands where I can find this specific coat that looks like this one or something like that. And then it's very good at finding those, versus the base model or the normal model will say it will give you some brands but it won't necessarily fit all of the constraints that I had given, like I want it to sell this fake fur coat that's this length this season or something. It's not going to be able to do that because it just won't have the up-to-date information and also just won't necessarily be able to deal with all of the constraints in a query in one shot. o1 isn't browsing as comprehensively. I'll use it to find things where I'm looking for a very specific thing that would take me hours to find. So I'm looking for this very specific item or sweater that is probably available on Real Real or somewhere but I can't find it, or I'm looking for an Airbnb with very specific constraints. So I think those kinds of things Deep Research is good for, and then more general, high-level things you should use normal search for.

是的。好吧，我承认我有一些多年的浏览/购物任务，我现在让 Deep Research 来做。我想再问一个经验问题：在训练 Deep Research 的过程中，有没有一个特别的成功或失败让你感到惊讶？

Yes. Well, I will admit I have had some multi-year browsing/shopping tasks that I am now making a crown job for Deep Research. I suppose I want to ask just one more experience question, which is: was there a particular win or failure that surprised you in the training of Deep Research?

这真的是那种我们认为在浏览任务上训练会有效的事情。我们对此有很好的信心，但实际上，当你第一次使用这个算法在新数据集上训练模型，看到它真正工作并与之互动时，即使我们认为它会有效，也还是相当不可思议。所以老实说，仅仅它工作得这么好就相当令人惊讶。尽管我们认为它会，如果这说得通的话。

It really was one of those things where we thought that training on browsing tasks would work. We felt like we had good conviction in it, but actually the first time you train a model on a new dataset using this algorithm and seeing it actually working and playing with the model was pretty incredible, even though we thought it would work. So honestly, just that it worked so well was pretty surprising. Even though we thought it would, if that makes sense.

是的。这是一种直观的体验，就像，哦，道路铺满了草莓之类的。

Yeah. It's the visceral experience of like, oh, the path is paved with strawberries or whatever.

完全正确。但有时它失败的地方也令人惊讶。比如有时它会犯一个错误，它做了如此聪明的事情，然后犯了一个错误，我就在想，你为什么这么做？停下来。所以我认为肯定还有很多改进空间，但到目前为止，我们对这个模型印象深刻。

Exactly. But then sometimes some of the things that it fails at are also surprising. Like sometimes it will make a mistake where it will do such smart things and then make a mistake where I just think, why are you doing that? Like stop. So I think there's definitely a lot of room for improvement, but yeah, we've been impressed with the model so far.

我习惯了我所有的技术工具都是即时的。Deep Research 不是即时的。它在思考和使用工具。

I'm used to all my technology tools being instantaneous. Deep Research is not instantaneous. It's thinking and using tools.

能更快吗？

Can it be faster?

是的，我确实认为存在一个很好的中间地带，有时你不想让它做非常深入的研究，但希望它比搜索做得更多。我认为我们很快就会发布一些东西，人们会对此感到满意，并且会填补这个空白。

Yeah, I do think there's a good middle ground in between where sometimes you don't want it to do really deep research, but you want it to do more than a search. And I think that we will release things soon that people will be happy about and will fill that gap.

好的。我不知道如何传达这种偏好，但我希望在某个时候能切换一下，比如‘尽你所能’，因为我会对一个人这么说。我希望你在接下来的五分钟内尽可能做到最好。

Okay. I don't know how to communicate this preference, but I want to like toggle at some point to be like as much work as I mean because I would say this to a human. I want you to do as good of a job you possibly can do in the next five minutes.

是的。你看，我认为让用户做这个决定实际上是一个糟糕的用户体验。模型应该更擅长知道需要思考多长时间。我认为我们在训练模型时做了一个决定，就是每次都追求最大思考时间。所以，我肯定有时会问一个非常简单的查询来测试，然后因为模型还在思考而感到相当沮丧。所以我确实认为这也是一个需要改进的领域，即知道思考多长时间。但是的，我猜想对于我们的深度研究，我们总是会专注于那些需要最长时间的任务，然后我认为像 O3 或下一个 O 模型会有一个更好的中间地带。

Yeah. See, that's something where I think it seems like a bad UX to actually make the user make that decision. The model should just be better at knowing how much time to think. I think we made a decision when training the model that we just are going to go for max thinking time every time. So, I'm sure I've asked it a really simple query sometimes just to test and then get quite frustrated that it's still thinking. So I do think that that's also an area for improvement is knowing how long to think for. But yeah I suspect with our deep research we will always be focusing on the tasks that take the maximum length of time and then I think like O3 or O next will have a better in between.

你能想象未来深度研究需要一整天才能完成的任务是什么？

What is an example of a task you can imagine deep research taking a day at in the future?

我的意思是，有些任务会让 GPU 冒烟。我认为任何任务，现在在五分钟或三十分钟内，它就能完成人类专家需要数小时才能完成的工作。所以我想，在一小时内，它可以完成人类需要数天才能完成的工作。在一天内，它可以完成人类需要数周才能完成的工作。显然，要让它扩展到那种程度会有很多挑战。但我认为你可以想象它完成一个原本需要数周才能完成的研究项目，或者写一篇论文之类的。

I mean there's some GPU smoking. I think anything that would take I mean right now in five or 30 minutes it can do what human experts rate take many hours. So I guess in an hour it could do something that would take a human days. In a day it could do something that would take a human weeks. Obviously there'll be a lot of challenges to get it to scale like that. But I think you can imagine it doing a research project that would have taken weeks to complete or like write a thesis or something like that.

好的，我要让我们的实习生和它竞争。好吧，那接下来几个月看看。

Okay, I'm going to make our intern compete with it. Okay, over the next couple months then.

是的，听起来不错。

Yeah, sounds good.

如果你展望一年后，这在 AI 领域是很长的时间，你认为有什么事情会让人们感到惊讶，即智能体能够做到并且实际上会发布？

If you were to project forward a year, which is a really long time in AI land, what is something that you think will surprise people that agents can do and that will actually be released?

所以，这要考虑到安全因素。一个通用的智能体，能够完成你在许多不同领域中的许多任务。比如对我来说，我经常编程。我希望有一个在编程方面相当胜任的智能体，我可以信任它，给它一个任务，它希望能创建一个拉取请求之类的，但也许我可以让同一个智能体帮我预订去韩国的旅行之类的。我希望我们能获得更统一的体验，但我也认为这些模型改进的速度会让大多数人感到相当惊讶。

So, it takes the safety considerations into account. A general agent that could do a lot of the tasks that you would do in a lot of different areas. Like for me, I do a lot of coding. I'm hoping that there'll be an agent that is pretty sufficient at coding, but that I will just trust to I'll give it a task and it will hopefully make a PR or something, but maybe I can ask the same agent to help me book a trip to Korea or something. I hope that we'll get to a more unified experience, but I also think that the rate at which these models are improving is going to be pretty surprising to most people.

为什么你认为统一的体验很重要，或者为什么你认为这有意义？

Why do you think a unified experience is important or why do you think that makes sense?

因为我认为今天的情况很不同，显然 ChatGPT 是一种非常全面的体验，但人们在不同场景下使用不同的模型，比如用于编程的下一行补全模型，感觉就像完全不同的环境。我认为你可能两者都需要。比如你可能想要一种体验，在某个时候你可以覆盖或打断模型，说‘哦，不，我不是那个意思。’或者你可以接管并开始输入一些东西。尤其是在短期内，因为模型在很多领域不如人类，而在其他领域更胜一筹。所以我认为这将是两者的结合：你让模型做某事，但以编程为例，你可能也在你的 VS Code 或 Cursor 中，它一直在为你做某事，但你也可以实际输入并自己写一些代码。所以我认为这将是这些事情的结合，但我有点希望它就像在 Slack 上有一个同事，或者一个远程同事，你可以让他们为你做事，给他们发一条 Slack 消息，然后他们就开始做。然后你可以审查他们的工作或在某个时候提供帮助，但这似乎是一个非常不错的通用界面，你不需要考虑应该让哪个智能体做哪个任务，你应该能够自己弄清楚。

Because I think today it's quite different to think about obviously ChatGPT is one experience that's very encompassing but there are models that people use in different contexts like next line completion type models for coding that feel like a very different setting. I think that you'll probably want both. Like you'll probably want an experience where you can at some point override or interrupt the model and say, 'Oh, no. I didn't mean that.' Or you can take over and start typing something. Especially in the short term as the models are not as capable as humans in a lot of areas and are more capable in other areas. So I think it will be a combination of you asking the model to do something, but then maybe with the coding example, you're also in your VS code or cursor, and it's been doing something for you, but you can also actually type and write some of it yourself. So I think it will be a combination of those things, but I kind of want it to be something that is just like having a coworker on Slack or a remote coworker you can just ask to do things for you, send them a Slack message, and then they'll start doing it. And then you can review their work or help at some point but it seems like a pretty nice general interface and you don't have to think about which agent should I ask to do which task like you should just be able to figure it out.

我对此的心理模型是，我的一般理念是，我实际上喜欢和我一起工作的人。综合考虑，我更喜欢和更少的人一起工作，管理开销更少，因为每个人都有更多的背景，我对他们有更多的了解。因此，一个普遍有用的智能体因为这个原因很有吸引力。而且你只需要告诉它一次，它就会记住，然后它会对你在做的所有事情都有状态，诸如此类。

The mental model I have for this is my general ethos is actually I love the people I work with. I prefer to work with fewer people with less management overhead all things considered because each person has more context and I have more understanding of them. And so like the universally useful agent is attractive for that reason. And you only have to tell it something once and it will remember and then it will have state on everything you're working on, things like that.

太棒了。嗯，这是一次很棒的对话，Isa。感谢你参加，也感谢你发布产品。

Awesome. Well, this has been a great conversation, Isa. Thanks for doing this and thank you for the product release.

非常感谢你邀请我，也感谢你使用深度研究。在 Twitter 上找到我们 @no prior pod。如果你想看我们的脸，请订阅我们的 YouTube 频道。在 Apple Podcasts、Spotify 或任何你收听的地方关注这个节目。这样你每周都能收到新剧集。并在 no-bers.com 上注册邮件或查找每集的文字记录。

Thank you so much for having me and thank you for using Deep Research. Find us on Twitter at no prior pod. Subscribe to our YouTube channel if you want to see our faces. Follow the show on Apple Podcasts, Spotify, or wherever you listen. That way you get a new episode every week. And sign up for emails or find transcripts for every episode at no-bers.com.