为人人构建持续学习产品：对话 Trajectory 创始人 Ronak

我们的产品哲学很多其实来自 Apple。在 2000 年代初，你有能上网的电脑，也有能打电话的手机。我觉得没人想到把两者结合起来，直到 Apple 说：‘好吧，如果把互联网放到手持设备上，把两者结合会怎样？’我认为通过这样做，基本上把训练、评估和部署放在一个地方，让每个人都能极其简单地操作，然后你可以闭环，基本上现在把模型上传回生产环境，收集新的轨迹。这突然成了一个神奇的范式，我认为每家公司很快都会采用。

A lot of our philosophy in product actually comes from Apple. In the early 2000s, you had a computer that had access to the internet and you had a phone that you could make phone calls. And I think no one really thought to put the two together until Apple said, 'Okay, what if you actually put the internet on a handheld device and combine the two?' And I think by doing this, by basically putting training, eval, and deployment in one place, making that just dead simple for every single person to do, and then you can close the loop and basically now upload that model back to production, gather new traces. It suddenly is this magical paradigm that I think every single company is going to adopt very soon.

Reach，感谢你加入我们。很高兴有你。我们很想深入探讨你正在构建的东西、你如何看待研究世界，以及你认为未来走向何方。也许先从持续学习开始，你们花了很多时间思考它。能为我们定义一下吗？我想你也说过这个领域处于伪持续学习状态。什么是真正的持续学习，我们如何知道何时达到真正的持续学习？

Reach, thank you for joining us. It's a pleasure to have you. We would love to dig into a bunch of what you're building, how you see the world of research, and where you think the future is going. Maybe to start, you guys spend a lot of time thinking about continual learning. Can you define it for us? I think you've also said this field is in a state of pseudo continual learning. What is real and how will we know when we get to the real thing?

这是个好问题。所以，当我思考持续学习时，我实际上认为人脑是一个很好的模型，是我们应该努力达到的目标。在某种意义上，我们是真正的持续学习者。我们在线学习。我们不断更新、迭代。而我认为我们现在所处的世界，有了智能体和 AI，就像你说的，是伪持续学习。我小时候，家里经常自驾游。我不知道你们怎么做，但那时我们会去 AAA，他们会给我们一堆地图。他们会标出所有东西，然后我们上路。如果途中想改变路线，我们得打电话给 AAA 或去实体店。我觉得这就是我们现在所处的世界：我们有这些惊人的智能体，它们非常智能，我们有这些映射算法，但你只能离线咨询几个地方，任何时候想更新路线，你都得做一次大规模批量更新。我们的第一个目标，也是我们在 Trajectory 构建的，首先是让这种访问民主化，对吧？甚至只是训练这些模型并为人们定制它们。我认为这有点像 MapQuest，突然之间它仍然离线，但至少我们让每个人不需要去 AAA，而是能够映射这些算法。但持续学习的最终阶段更像 Google Maps，每次你开车，你走的每条路线，它都在更新。它在学习，聚合最快路线和交通状况。那是我想要达到的世界。

That's a great question. So, when I think about continual learning, I actually think the human brain is a really good model for where we should aspire to get to. In some sense we are true continual learners. We learn online. We are constantly updating, iterating. And I think the world we're in right now with agents and AI is kind of like you said, pseudo continual learning. When I was growing up, my family used to do a bunch of road trips. And I don't know how you guys did things, but we would go to AAA back in the day and they would give us a bunch of these maps. They would highlight everything, and then we'd go on our road trip. And if there's any point where we wanted to change our route, we'd have to either call AAA or go to a physical location. And I feel like that's the world we're in right now where we have these amazing agents, they're very intelligent, we have these mapping algorithms, but you can only consult offline a couple of places, and anytime you want to update your course you have to just do an entire mass batch update. What the first goal and what we're building at Trajectory is first the ability to just democratize that access, right? To even just training these models and customizing them for what people want. I think of that as kind of like MapQuest, where suddenly it's still offline, but we at least got the ability for everyone to not need to go to AAA, but get the ability to map these algorithms. But the final stage of continual learning is something more like Google Maps, where every single time you're driving, every route that you make, it is updating. It's learning, aggregate what the fastest route is and where the traffic is. That's the world I want to get to.

这个类比真好。我们最近请了 Rich Sutton 和 David Silver 上节目，他们都评论说持续学习在当今架构下不太可能。我很好奇你的观点：什么是可行的，什么是不可行的，现状如何？我们需要重大的架构飞跃来实现它吗？我们今天处在什么位置？

That's a really good analogy. So we had recently we had Rich Sutton and David Silver on the show and both of them kind of made the comment that continual learning isn't really possible with today's architectures. I'm curious in your point of view what's doable, what's not doable, what's the state of play? Do we need major architectural leaps to kind of bring this to fruition? Where are we today?

是的。所以，有趣的是，当你想到 Transformer 和 LLM 时，我觉得很多人可能看过 Karpathy 的 tiny GPT 视频之类的，基本上你有一堆注意力块和 Transformer 层层堆叠。很多 LLM 的 Scaling 就是把这些 Transformer 堆叠起来。但当你看看任何论文，比如最近刚出的 DeepSeek v4.1 flash，他们的架构实际上有点 lambdized 或改变了很多基础架构，比如权重内有快慢记忆。你有第 2 层和第 50 层之间的跳跃连接。所以我认为这些架构变化实际上已经在基础模型中发生了。我同意，这将是持续学习的关键组成部分。我想做的是利用所有在架构层面发生的技术和研究，但然后真正连接到现实世界，因为归根结底，任何 AI 进步的真正价值，尤其是持续学习，是人们如何在现实世界中使用模型。我认为公司心中有‘什么看起来好’的想法，而整个循环中最难的部分是将其转化为模型的实际改进。我认为这就是缺失的层面。

Yeah. So, it's funny because when you think about a transformer and LLMs, I feel like a lot of people have seen maybe Karpathy's tiny GPT video or something where you basically have the stack of attention blocks and transformers that are built on top of each other. And a lot of LLM scaling has just been stacking these transformers on top of each other. But when you look at any paper like DeepSeek v4.1 flash that just came out recently, their architecture is actually kind of lambdized or changed a lot of that base architecture where you have things like fast and slow memory within the weights. You have skip connections that are going between layer 2 and layer 50. And so I think these architectural changes are actually already happening within the base models. And it will be a key component, I do agree, for continual learning. Where I want to sit is basically leverage all of that technology and research that's happening at the architectural level but then actually connect the dots to the real world because at the end of the day the real value of any AI progress but certainly continual learning is how are people actually using models in the real world. And I think companies have this idea of basically what good looks like in their mind and the hardest part of this entire loop is then translating that to actual improvements for the model. I think that's where that missing layer is.

是的。那么你如何连接到现实世界，其中多少是产品化或可产品化的，多少只是手把手帮助人们完成过程？

Yeah. So how do you connect the dots to the real world and how much of that is product or productizable and how much of that is just handholding and helping people through the process?

是的。所以到目前为止，从数据到智能这个事实一直很难。到目前为止，这只有在实验室里才能做到。你需要一个庞大的研究团队。所以过去几年，我们开始有垂直公司进入这个领域，并建立庞大的研究团队。我之前在 Windsurf，我们踏上了这个旅程并训练了一个模型。Cursor 也用 Cursor Composer 做了同样的事，但问题是你需要数亿美元的研发算力。你需要一个真正庞大的研究团队来投资，然后你还需要大量时间。所以我们花了六到九个月才真正达到可以训练前沿模型的地步。那还是有一种手把手咨询模式。那仍然是我们所处的世界。我认为任何好产品都存在，当世界上每家公司都在一遍又一遍地做同样的事情。如果你能把它打包成一个漂亮的界面，只说这些是你关心的旋钮和杠杆，用来转化‘什么看起来好’，然后其他一切都被打包隐藏。我认为那是一个漂亮的界面。

Yeah. So up until now it's been really hard to just go from this fact of like data to intelligence. Up until now that's only been in the labs. And you need a massive research team. So in the last couple years we started to have vertical companies going into this space as well and building a massive research team. So I was previously at Windsurf where we embarked on this journey and trained a model. So did Cursor as well with Cursor Composer but the problem was you needed hundreds of millions of dollars of R&D in compute. You needed a really massive research team to invest in and then you also needed a lot of time. So it took us six to nine months to actually get to that point where we could train a frontier model. And that is with a kind of handholding sort of consulting model. That's still sort of the world that we live in. I think any good product exists when you have every single company in the world trying to do the same thing over and over and over again. And if you can package that into a really beautiful interface that just says here are the knobs and levers that you care about to translate that what good looks like, and then everything else is packaged away. I think that's a beautiful interface to do that.

你和你的联合创始人在 Windsurf 时看到了什么，他们也在做模型，帮助你意识到嘿这是可重复的，有产品可以构建。你什么时候有了那个顿悟时刻和那个认识？

What did you and your co-founders see when you were at Windsurf and when they were working on models as well to help you realize hey this is repeatable that there is a product to be built. When did you have that aha moment and that realization?

是的。

Yeah.

所以部分原因是，你知道，我们在生产环境中训练这些模型，基本上是在说产品中发生的所有这些编辑、重试是什么，并从中学习，在 Windsurf 时就是这样。然后结合这一点，我的联合创始人——Michael 也在 DeepMind，他当时在做 Gemini 机器人，他们也开始看到同样的事情：你实际上可以把这些机器人部署到现实世界，构建持续学习基础设施，基本上让这些机器人每天都能更新。然后 Arjun 在 Apple 做 Vision Pro。同样，他基本上在说我们如何利用艺术家反馈，并将其用于多模态基础模型。所以以某种方式，我们所有人基本上都在解决这个宏大的对齐问题：你如何真正从现实世界中获取反馈并改进模型。

So part of it was, you know, we were training these models on production, basically saying what are all these edits, the retries that are happening in the product, and learning from that at Windsurf. And then combined with that, my co-founders—Michael was at DeepMind as well, he was working on Gemini robotics, and they're starting to see the same thing where you could actually deploy these robots in the real world, build continual learning infrastructure to basically allow these robots to update every single day. And then Arjun was at Apple working on the Vision Pro. And the same thing, he was basically saying how can we take artist feedback and use that for multimodal foundation models. So in one way or another, all of us were basically figuring out this grand alignment problem of how do you actually take this feedback from the real world and improve the models.

问题在于我们都在这些大型研究实验室里，对吧？所以我开始意识到，好吧，持续学习将成为下一个学习范式。有没有办法让我在实验室内部真正做到这一点？我很快意识到的是，在 DeepMind，我们的全部焦点是训练一个巨大的模型，然后将那一个模型分发给世界上的每个人，这显然在整个行业一直是一个成功的商业模式。但如果你想基本上拿一个模型，为单个人定制，然后只分发给那个客户——比如我们的平台与 Harvey 合作所做的那样——那对于大型实验室来说，仅仅为一个客户就要构建大量的基础设施。

The problem is we were all at these big research labs, right? And so I started to realize, okay, continual learning is going to be the next learning paradigm. Is there a way that I can actually do this within the labs? And the thing I very quickly realized is at DeepMind, our entire focus was training one massive model and then distributing that one model to everyone in the world, and that's been a winning business model obviously across the board. But if you want to basically take one model, customize that for a single person and then distribute that one model just for that customer—like for example what our platform does with Harvey—that is an immense amount of infrastructure for a massive lab to build just for one customer.

是的。也许可以以 Harvey 为例，谈谈你们做了什么。

Yeah. Talk about what you guys have done with Harvey maybe as an example.

所以 Harvey 基本上——他们直到现在都建立在前沿闭源模型之上，对吧？他们能够在此基础上构建令人难以置信的界面和产品，帮助世界上大量的律师事务所。但他们意识到的是，他们积累了大量的数据知识、所有这些细微差别，他们实际上能够看到律师事务所真正关心什么，以及在这个领域工作 30 年积累的经验。而使用前沿模型，他们无法真正利用这些数据并使其对自己有利。所以他们意识到的是，嘿，好吧，他们能够构建，例如，一个法律智能体基准，以及他们发布的内容，并利用他们拥有的所有专家信息，然后如果他们获得权力，基本上利用这些数据来训练模型，那么他们就能塑造自己的命运。所以通过 Trajectory，他们能够做的是训练一个便宜六倍的模型，并且将开源模型的准确率从大约 4% 提升到大约 18%。这仍然有很大的增长空间，显然他们能够做的，但现在突然获得了研究实验室的力量，而且最棒的是这些模型最终比前沿模型便宜六倍，因此你能够消耗更多的 token 来完成这些任务。

So Harvey basically—they were up until now riding on top of the frontier closed source models, right? And they've been able to build incredible interfaces and products on top of that that help tons of law firms in the world. But what they realized is they have an immense amount of knowledge of data, of all of this nuance built up that they are actually able to see what the law firms actually care about and the built-up experience of just 30 years of working in this domain. And with using frontier models, they're unable to actually take that data and use that to their advantage. And so what they realized is, hey, okay, they're able to construct, for example, a legal agent benchmark, and what they released and using all of the expert information that they have, and then if they get the power to basically use that data to train a model, then they get to shape their own destiny. So with Trajectory, what they were able to do is train a model that is six times cheaper and it goes from around 4% accuracy for an open source model to around 18%. And this is—there's still so much room to grow obviously what they're able to do, but now suddenly get the power of a research lab and on top of that the nice part is these models end up being six times cheaper than the frontier and so you're able to expend so many more tokens to achieve these tasks.

所以你们选择了 Trajectory 这个名字，因为它与这个持续学习概念相关。你们是怎么选这个名字的，并告诉我们它如何契合你们正在做的事情。

So you picked the name Trajectory as it's related to this continual learning concept. How did you pick the name and tell us how it fits into what you guys are doing.

是的，所以很有趣，因为现在世界非常关注 token，我认为很多人将 token 视为工作单元。但如果你想想我们与其他人类互动的方式，我并没有在数你实际说了多少词，对吧？而与同事合作时，我们很多时候考虑的是一个任务或一条轨迹。而这正是我真正想要达到的工作单元。在强化学习中，它基本上与 rollout 或你在训练栈中实际做的事情同义。而在持续学习中，你有一个强化学习训练栈，就像 rollout 引擎和训练器。rollout 引擎就像在模拟中，基本上做一堆 rollout，并在现实世界中为持续学习做这些轨迹。你的 rollout 引擎就变成了现实世界中的推理和 token。而现在突然你的轨迹就是正在发生的具有经济价值的任务。所以我希望这种范式传播开来，我们考虑的工作单元是轨迹。除此之外，我认为对于不了解强化学习的人来说，也非常好的一点是，我们基本上让公司拥有那条路径并真正自行成长。所以，我认为它有一个不错的双关含义。

Yeah, so it's funny because the world right now is very focused on tokens and I think a lot of people view tokens as a unit of work. But if you think about the way that we interact with other humans, I'm not counting the amount of words that you're actually talking about, right? And with co-workers, we a lot of the time are thinking about a task or a trajectory. And that's the unit of work that I really want to get to. In RL, it basically is synonymous with a rollout or what you would actually do in a training stack. And with continual learning, you have an RL training stack that's like a rollout engine and a trainer. A rollout engine is just like in a simulation basically like doing a bunch of rollouts and doing these trajectories in the real world for continual learning. Your rollout engine becomes just the inference and the tokens in the real world. And now suddenly your trajectories are the tasks of economic value that are happening. And so I want this basically paradigm to propagate where the unit of work that we think about is a trajectory. Beyond that, I think for also people that don't understand RL, it is also super nice that we are basically letting companies just own that path and really grow on their own. So, I think it has this nice double entendre.

Kashi，我喜欢这个将轨迹视为具有经济价值的工作单元的想法。你也几次谈到了“好的样子”，这听起来像是评估或基准。持续学习的核心原语是什么，或者关键要素有哪些？

And Kashi, I like this idea of a trajectory as a unit of economically valuable work. You've also a couple times talked about what does good look like, which kind of sounds like an eval or a benchmark. What are the core primitives for continual learning or what are the key ingredients that kind of go into this?

是的。所以我们在 Trajectory 的目标实际上是限制这些要素，使其真正只是，你知道，好的样子。然而，现在公司需要很多组件，而我们作为产品也试图弥合这一差距，并使其尽可能容易。所以现在作为一家公司，你需要几样东西。第一，你需要能够追踪生产环境中发生的所有轨迹。有很多 AI 原生公司经历了这种转变，他们不仅记录主智能体调用，还记录所有子智能体、产品中发生的事情以及用户给出的任何反馈。理想世界是你获得所有这些事情的完美时间线，以便你可以追踪它们。所以这是第一步。

Yeah. So our goal at Trajectory is to actually limit these ingredients so that it is literally just, you know, what good looks like. However, right now there are a lot of components that companies need and then we as our product as well try to bridge that gap and make that as easy as possible. So right now as a company you need a couple things. One, you need to be able to trace all of your trajectories that are happening in production. And there's lots of AI native companies that have gone through this transformation where they're logging not just the main agent call but also all the sub agents, what's happening in the product as well, any feedback that the user is giving. The ideal world is you get this kind of perfect timeline of all of those things so that you can track them. So that's the first step.

就这一点而言，现在人们是怎么做的？比如那些追踪记录存在哪里？

And just on that, what are people doing that in today? Like where do those traces live?

是的，在我看来，我认为整个持续学习生态系统需要很多参与者来构建所有这些基础设施。所以有些公司在内部做这件事，但也有很多很棒的产品，比如 LangChain、Brain Trust、Raindrop，你可以基本上接入你的基础设施或生产智能体，然后追踪所有这些事情。所以我们发现很多 AI 原生公司会使用很多这些产品来完成第一步。

Yeah, in my view I think the entire ecosystem of continual learning needs to be a lot of players that basically build up all this infrastructure. So some companies are doing this in-house, but there's also a lot of great products out there like LangChain, like Brain Trust, Raindrop, where you can basically plug into your infrastructure or your production agents and then basically trace all of these things. So what we're finding is a lot of AI native companies will use a lot of these products in order to get that first step out of the way.

好的，所以我们有了追踪记录，然后呢？

Okay, so we got traces and then what?

所以那是追踪记录。第二部分是让这些可回放。

So that's traces. The second part is then making these replayable.

你需要某种基础设施来做到：给定我在生产环境中看到的一条轨迹，我如何能实际回放它来进行评估，看看什么才是好的表现，或者现在也用于训练。所以这也是公司需要投入一些时间的地方。我们在产品中让弥合这一差距变得非常容易。所以如果你带来你的轨迹，并说，好的，这是我可能想要的环境概念——用更技术性的术语来说，就是一个 Docker 文件或某种系统需求——那么你基本上可以回放整个轨迹。所以这是第二部分。然后第三部分是实际的修改或算法。我认为这就是作为一家公司内部很难做到的地方：整个训练栈、整个损失函数、奖励函数，所有这些需要构建起来以将这些任务转化为模型实际改进的组件。所以这就是我们想要一开始就抽象掉的主要部分。

You need some sort of infrastructure to say, okay, given a trace that I've seen in production, how can I actually replay that for an eval to see what good looks like, or right now for training as well. And so that's where companies also need to invest some time. What we make really easy in our product is to bridge that gap. So if you bring your traces and you say, okay, here's the concept of an environment that I might want—in more technical terms, it's a Docker file or some sort of system requirements—then you can basically replay that entire trace. So that's the second part. And then the third part is the actual modification or the algorithm. And I think that's where it's really difficult to do this in-house as a company: the entire training stack, the entire loss function, the reward function, all of these components that need to be built up to translate these tasks into actual improvements for the model. And so that's the main part that we want to abstract away off the bat.

关于你提到的抽象掉一些部分，到目前为止已经抽象了什么，接下来是什么，你知道，更远一些的，比如这个产品如何随着时间的推移而整合起来？

To your point on abstracting away some of that, what has been abstracted thus far, what's next, you know, what's kind of further out like how does this product come together over time?

现在主要是你带来轨迹和奖励，以及一些环境的概念。这里需要大量手动工作的两个部分是环境——实际构建与真实生产相匹配的东西。而且很多时候实际上不可能创建一个完美的环境，对吧？然后第二部分是奖励。很多时候，法律工作流或投资银行工作流并没有完美的可验证奖励可以用来训练，对吧？所以这是我们在研究方面真正推进的两个领域。所以对于环境，有一个叫做 GRPO 的算法。这是目前存在的主要后训练算法。你需要做的是，基本上想象你有一个玩家在视频游戏中，它基本上在到处跳，在一堆平台上跳来跳去，然后最终你得到一些奖励。在 GRPO 中需要的是基本上比较相同的任务、相同的游戏关卡，你需要玩八次或 16 次，我们称之为一个组。现在的问题是，对于人类来说，很多时候我们不需要一遍又一遍地重放完全相同的环境。我们只是在世界中体验并进行一次 rollout。所以目标是，我们能否修改 GRPO 算法，达到我们所谓的组大小为 1 的世界，而不是需要一组 16 个。所以我们在这里取得了很大进展，现在如果你能取 10,000 条轨迹，基本上它们都来自不同的任务，我们实际上可以从那一个行为或那一次 rollout 中学习。所以这是我们实际上发布了一篇博客文章的算法，进入这个组大小为 1 的范式，这样实际上消除了对环境以及所有需要构建的基础设施的需求。第二部分是奖励。所以这是另一个因素，假设我再次有 10,000 条轨迹。默认是我需要去标记每一条轨迹，说这是可验证奖励或这是好的表现。这是可行的。只是工作量很大。所以现在我们的目标是如何实际减少所需的时间，将这种好的表现的概念用少得多的功作完成。所以我们意识到，假设我有 10,000 条轨迹。如果我能智能地选择其中的一百条，基本上代表所有这更广泛的 10,000 条，然后让客户或用户基本上密集地说，好的，这是好的表现，这是仅针对那 100 条轨迹的评分标准或可验证标准。然后我们可以做的是基本上训练一个元评分器，从这 100 条的行为中学习到更广泛的 10,000 条。有了这个，你得到一个非常干净的界面，允许你只需在整个板上传播这个奖励。所以我认为显然在这个方向上它不是完美的解决方案。理想的目标实际上是，你有某种可观测性界面，你可以说这是我的智能体擅长的五件事，这是它不擅长的五件事。我只需圈出那五件事并点击训练。这就是我想要达到的世界，这就是我们推动研究的方向。

Right now it's mainly you bring traces and reward and some notion of an environment. The two parts here that require a bunch of manual work is the environment—actually constructing this thing that matches the real production. And a lot of the time it's actually impossible to create a perfect environment, right? And then the second part is reward. A lot of the times a legal workflow or an investment banking workflow doesn't have some perfect verifiable reward that you can train off of, right? And so these are the two areas that we're really pushing on the research side. So for the environment, there's an algorithm called GRPO. That's the main kind of post-training algorithm that exists right now. What you need to do there is you basically like imagine you have a player that's in a video game and it's basically going around and hopping around a bunch of platforms and then at the end of the day you get some reward. What's needed here in GRPO is basically comparing like the same task, the same level of the video game and you need to play that out eight times or 16 times and we call that a group. Now the thing is with humans a lot of the time we don't need to replay the exact same environment over and over again. We're just in the world experiencing and doing one roll out. And so the goal is could we modify GRPO as an algorithm to get to a world where it is what we call group size one instead of needing a group of 16. And so we've made a lot of strides here where now if you can take 10,000 trajectories and basically they're all from different tasks, we can actually learn from just that one behavior or that one roll out. And so that's one algorithm that we actually released a blog post about is going into this group size one paradigm and so that actually eliminates the need for an environment and all of this infrastructure that needs to be built out. The second part is for reward. And so this is another factor where let's say I have again 10,000 traces. The default is that I need to go and label every single one of those traces to say here's verifiable reward or here's what good looks like. And this is tractable. It's just a lot of work. And so now our goal is how can we actually reduce that amount of time needed to take this notion of what good looks like and do that in much less amount of work. So what we've realized is let's say I have 10,000 traces. If I can smartly select a hundred of them that are basically representative of all of these broader 10,000 and then have the customer or the user basically densely say okay here's what good looks like, here's a rubric or verifiable criteria for just those 100 traces. And then what we can do is basically train a metagrader that learns from those behaviors of the 100 to the broader 10,000. And with that you get a really clean interface that allows you to just propagate this reward across the board. So I think obviously it's not the perfect solution in that direction. The ideal goal actually is that you have some sort of observability interface where you're like here is the five things my agent's good at, here's the five things it's bad at. And I just circle those five things and hit train. That's the world I want to get to and that's where we're pushing that research direction.

非常酷。所以你刚刚发布了一个产品来做一些这方面的工作。告诉我们吧。你在产品中发布了什么，它是给谁用的？

Very cool. So you have a product that you just released to do some of this. Tell us about it. What are you releasing in the product and who is it for?

是的，到目前为止，训练这些模型需要庞大的研究团队、大量时间和大量算力。我们想要做的就是打造最简单、最直观的界面来训练模型，并根据你关心的内容进行定制。所以我们发布的平台 trajectory 基本上是一个非常简单的系统，你可以带上你的智能体轨迹和好的表现。它是一个一键训练界面，只需几个智能体提示，然后你可以评估模型，然后在一个地方部署。我认为真正神奇的部分是——我们在产品中的很多哲学实际上来自 Apple。在 2000 年代初期，你有一台可以访问互联网的电脑，你有一部可以打电话的手机，我认为没有人真正想到将两者结合起来，直到 Apple 说，好吧，如果你真的把互联网放在手持设备上并结合两者呢。我认为通过这样做，基本上将训练、评估和部署放在一个地方，让每个人都能做到极其简单，然后你可以闭环，基本上现在将模型上传回生产环境，收集新的轨迹。这突然就是这种神奇的范式，我认为每个公司都会很快采用。

Yeah, up until now it's taken a massive research team a ton of time and a ton of compute in order to train these models. What we wanted to do is just make the easiest and most intuitive interface to train models and customize them for what you care about. And so trajectory, the platform that we released, is basically a very simple system where you can take your agent traces and what good looks like. It's a one-click train interface that just takes a couple agent prompts and then you can eval the model and then deploy it all in one place. And I think the really magical part about this—a lot of our philosophy in product actually comes from Apple. And when in like the early 2000s you had a computer that had access to the internet and you had a phone that you could make phone calls and I think no one really thought to put the two together until Apple said okay what if you actually put the internet on a handheld device and combine the two. And I think by doing this of basically putting training eval and deployment in one place, making that just dead simple for every single person to do and then you can close the loop and basically now upload that model back to production, gather new traces. It suddenly is this magical paradigm that I think every single company is going to adopt very soon.

非常酷。这个产品是针对谁的？比如你们今天的理想客户是谁？

Very cool. Who is the product targeted at? Like who is the ideal customer for you guys today?

是的。所以我们想要做的是让那些已经整理好智能体框架的公司变得非常简单。他们在这方面花了很多时间。但下一步基本上是修改最有价值的部分，即模型——让那变得超级超级容易。所以现在这个产品主要面向 AI 原生公司，比如 Harvey 和 Rogo，还有一些更大的公司如 Nvidia 已经在使用这个平台。但接下来的步骤是，我还想超越模型本身。所以另一个重要的部分是框架，因为我认为模型能给你效率。

Yeah. So what we want to do is make this very simple for companies that have already their agent harness in order. They've spent a lot of time there. But that next step of basically modifying the most valuable part which is the model—making that super super easy. So right now what the product is meant for is mainly AI native companies like Harvey and Rogo and some larger companies as well like Nvidia are already using the platform. But the next step after this is I want to also move beyond just the model. So another important piece is the harness as well because I think the model gets you efficiency.

它给你一个已经见过某个任务上百次或上千次的模型，让任务执行更快、更便宜、更可靠。而 harness 还能带来新的能力，那是以前不可能做到的。我认为很多正在采用 AI 的大公司，还没有做优化 harness 的工作——无论是为子智能体优化，还是说榨干前沿 token 的最后一点价值。所以最终的世界可能是：你用像 Fable 这样的前沿模型处理 10% 的任务，剩下 90% 的时间，你可以智能地路由到一个你训练过的模型，它效率高得多，也能带来更好的利润率。

It gets you a model that has seen a task a hundred times or a thousand times over, and making that faster, cheaper, and more reliable. What a harness gets you also is new capabilities. It's just something that was not possible before. And I think a lot of larger companies that are adopting AI haven't done the work to optimize the harness for sub-agents or for basically squeezing out that last mile even out of the frontier tokens. And so the eventual world is maybe you use a frontier model like Fable for 10% of tasks. And then for 90% of the time you can smartly route to a model that you've trained that is far more efficient and also gets you better margins.

跟我们讲讲你们目前合作过的一些客户的故事吧。你刚才稍微提到了 Harvey，或许可以再举几个例子。

Tell us some stories about some of the customers you've worked with so far. You gave us a little hint at Harvey, but maybe a couple more examples.

好的。这里一个非常有趣的例子是 Rogo。我认为 Rogo 正在采用一种非常有趣的智能体范式，我觉得这会成为未来。现在人们使用智能体的方式非常同步化，对吧？你给智能体一个提示，它去执行任务，然后返回结果。Rogo 非常感兴趣的不是那种冗长的模型——那种会吐出一堆垃圾的模型。他们想要的是一个非常简洁清晰的金融智能体，真正做你关心的工作。它可以在后台自主执行任务，但返回结果时非常高效。所以 Trajectory 的目标基本上就是：对 Rogo 来说，他们想要的是智能密度。他们希望模型能完成同样的任务，但比前沿模型高效得多。他们有数据来做到这一点，所以借助 Trajectory，他们能够训练出一个模型，在 token 效率上高出数倍，同时也更便宜、更快，而且还能在他们关心的指标上持续提升。

Yeah. So, one really interesting one here is Rogo. And I think Rogo is taking a very interesting paradigm to agents that I think will become the future. So right now the way people use agents is very much like synchronous, right? You prompt the agent, it goes and does its thing, and then it comes back. What Rogo is very interested in is not just like this kind of verbose model that will go and basically spit out a bunch of garbage, right? What they want is just like a very concise and clear finance agent that basically does the work that you care about. It can autonomously do things on the side, but then it comes back and is very efficient. And so what the goal of Trajectory is is to basically say, okay, Rogo, what good looks like for them is they also want the idea of intelligence density. They want the idea that a model can do the same task but do that far more efficiently than the frontier models. They have the data to do that, and so with Trajectory what they were able to do is train a model that is several times more efficient in terms of its tokens and also happens to be cheaper and faster, but then is also able to gain and climb on the things that they care about.

非常酷。好的，我们之前稍微聊了聊 GRPO 中组大小为 1 的情况。接下来呢？持续学习将走向何方？

Very cool. Okay, so we talked a little bit about like group size one with GRPO. What's next? Like where is continual learning headed?

好的。我们目前讨论的很多内容，主要只是帮助这些公司把他们认为好的标准转化为模型的改进。我认为要达到这样一个真正的世界——人类从现实世界中学习极其高效，并且对世界有内在的好奇心——我想说这是让我们实现智能体通过持续学习自主学习的缺失环节之一。现在的模型，它们执行任务，但并不真正积累任何经验或学习关于现实世界的任何东西。而如果你想想，比如说我扔橄榄球，对吧？我觉得作为人类，也许如果我以前从未见过橄榄球，我完全不知道会发生什么。但我扔了一百次之后，突然就建立了这种启发式。重要的是，我认为人类就像孩子一样，不断探索世界。我们看到足球，看到橄榄球，就尝试各种东西来建立这个心智模型，这让我们变得极其智能。而对于智能体，目前这还不存在。所以我们做了很多思考：如何真正量化好奇心。我认为一个有趣的思考方式是，强化学习中有个概念叫马尔可夫决策过程，基本上你可以把世界描述为一堆状态-动作对。状态就像我手里拿着球，动作是我把它扔出去，然后下一个状态是球最终落在哪里。现在 LLM 非常擅长的是直接从状态零到状态一。它们是下一个词预测器，对吧？但缺失的是中间部分：当你从 S0 到 S1 时，那个转移函数是什么？发生的转移函数是什么？LLM 目前并没有真正编码这个的方式。所以我认为有趣的是，人类非常清楚自己不知道世界的转移函数是什么，我们会说我要试着弄清楚这个转移函数是什么，我认为这就是好奇心的来源。所以如果有一种方法能基本上量化：这是我现在不知道的整个世界，而我作为智能体的目标是弄清楚如果我做这个任务会发生什么，我建立这个心智启发式，那么我认为我们就开始解决这个好奇心问题，并走向一个可以自主且持续地从世界学习的世界。

Yeah. So, a lot of what we've been talking about right now is mainly just helping these companies translate over what good looks like for them into improvements for the model. I think to get to this true world where humans are extremely efficient about learning from the real world and we're able to have inbuilt curiosity as well about the world, and I would say that's kind of one of the missing pieces to get us to agents actually learning autonomously on their own via continual learning. The models right now, they do a task, but they don't actually accumulate any experience or learn anything about the real world. Whereas if you think about, let's say I'm throwing a football, right? I feel like as a human, maybe if I've never seen a football before, I actually have no idea what's going to happen. And but I do that a hundred times over and now I suddenly build this heuristic, right? And the important part is I think with humans, like as kids we are constantly exploring the world. We see a soccer ball, we see a football and we kind of just try out things to build this mental model and that makes us incredibly intelligent. Whereas with agents that doesn't exist right now. And so we've done a lot of thinking on how do you actually quantify curiosity. And I think one interesting way to think about this is, so there's this idea in RL called a Markov decision process, and it's basically you can characterize the world as a bunch of state-action pairs. So a state is like I'm holding the ball in my hand, and then an action is I throw it, and then the following state is where does the ball end up. What LLMs right now are really good at is basically going straight from state zero to state one. They're next-token predictors, right? But what's missing is that intermediate part of what is that transition function when you actually go from S0 to S1? What is that transition function that happens? And LLMs don't really have a way of encoding that right now. So I think the interesting thing here is with humans we are very aware of when we don't know what a transition function is in the world and we say I'm basically going to try to figure out what this transition function is, and I think this is where curiosity comes in. And so if there's a way to basically quantify here's all of the world that I don't know about right now and my goal as an agent is to figure out if I do this task what happens, I build this mental heuristic, then I think we are starting to solve this curiosity problem and get to a world where you can autonomously learn and continually learn from the world.

你如何确保智能体的好奇心与用户的目标对齐？

How do you make sure that an agent's curiosity is aligned with a user's objectives?

所以，我认为这实际上是我非常热衷于研究持续学习的原因之一。目前研究似乎有两个方向。一个是 RSI，递归自我改进，另一个是持续学习。这两个概念非常紧密地联系在一起，或者说非常相似。它们本质上都是关于智能体如何从经验中学习。问题是，RSI 是把一个模型放进黑箱，基本上让它成为最好的 AI 研究员本身，以便它能自我改进，对吧？但这有点可怕，因为它恰恰引出了一个问题：这些模型和智能体真的会对齐并考虑人类的最佳利益吗？而我认为持续学习采取了一种非常不同的方法。它说我想从经验中学习，但是从与人类实际互动的经验中学习，然后在人类设定的、他们想要做的任务上变得更好。我认为关键区别就在这里：你需要允许这些模型并构建算法，使其保持人类在环。我不想最终走到智能体失控的世界，我认为做到这一点的方法就是坚持在这条对齐之路上。

So, I think that's actually one of the reasons why I'm very passionate about working on continual learning. So, there's two kind of directions that it seems like research is going right now. There's RSI, recursive self-improvement, and there's continual learning. And both of these are very, very tightly tied together, right? Or very similar concepts. They're essentially just how does an agent actually learn from experience. The thing is RSI is all about taking a model, putting it in a black box and basically making it the best AI researcher itself so that it can improve itself, right? But this is kind of a scary world, right? Because it exactly begs the question of are these models and agents actually going to be aligned and have the best interests of humans in mind. Whereas I think continual learning takes a very different approach. It says I want to learn from experience but learn from the experience of actually interacting with humans and then getting better at the tasks that they set out and that they want to do. And I think that's where the key difference is: you need to allow these models and build the algorithm so that it keeps humans in the loop. I don't want to end up in a world where these agents are going rogue, and I think the way to do that is to keep on this alignment path.

你们在 Trajectory 的产品和研究方面做了很多独特的事情，把这两半结合在一起。请多讲讲在 Trajectory 做东西的独特之处，以及你们聚集了哪些人来实现它。

You guys have done a bunch of unique things across product and research at Trajectory to bring these two halves of the whole together. Tell us more about what makes building at Trajectory unique and who are some of the people you bring together to make it happen.

好的。

Yeah.

所以我们所处的酷炫之处在于前沿研究与产品的交汇点，我们必须修改核心的强化学习算法，才能达到一个既能真正从世界中学习，又能将其打包成漂亮简洁的界面供人们训练这些模型的地方。

So the cool part that we are sitting in is the intersection of frontier research and product, where we have to modify the core RL algorithms in order to get to a place where you can actually learn from the world but also package that into a really beautiful and clean interface for people to train these models.

我们组建的研究团队中，有些人曾参与过，比如 Gemini 3 和 Gemini 机器人的联合创始人，其他一些人参与过 OpenAI 的 GPT-4o，以及在 MSL 构建 Muse 基础设施。但在产品方面，也有在 Stripe 构建支付 SDK 可扩展基础设施的人，还有 Netflix 最年轻的设计工程师，这类人。我认为他们之间的交集正是神奇之处开始发生的地方。

And so some of the research team that we've built had contributed to, like for example, the co-founders to Gemini 3 and Gemini robotics, other folks to GPT-4o at OpenAI, and building Muse infrastructure at MSL. But then on the product side as well, having people that have built scalable infrastructure at Stripe for payment SDKs, also we have the youngest design engineer at Netflix, these sort of people. And I think the intersection between them is where the magic starts to happen.

所以我们实际组建团队的方式是，通常你会有一个研究栈，那是公司某个角落里的意大利面条式代码库，然后你有一个构建得相当完善的产品。我们所做的是实际上将训练用的代码库也生产化了。

So the way we've actually structured the team, normally you have like a research stack that's the spaghetti codebase in one corner of the company and then you have the product that's kind of well built out. What we've done is actually productionize the code base for training as well.

这意味着微服务、Kubernetes 扩展、训练器每个部分的自动扩展、rollout 引擎。然后在设计方面，我们的设计工程师一直在启动训练运行，并深入理解研究体验是什么。我认为这是一个非常有趣和酷炫的领域。

So this means microservices, Kubernetes scaling, autoscaling of every single part of the trainer, the rollout engine. And then on the design side as well, our design engineers are kicking off training runs all of the time and understanding deeply what the research experience is. I think this is a really interesting and cool place to be in.

这其中很多也只有借助前沿编码智能体才可能实现。实际上现在我们大部分时间都在白板上讨论。我们会花很多时间争论系统设计。然后实际实现就是一个智能体的软件工厂，它们会去实际实现它。

A lot of this is also only possible because of frontier coding agents. Actually most of the time now we are spending our time whiteboarding. And we'll spend a lot of time just arguing about system design. And then the actual implementation is just a software factory of agents that will go and actually implement it.

所以在这里构建产品并不容易。在持续学习的世界里，产品构建者的角色是什么？

So building a product here is not easy. What is the role of a product builder in a world of continual learning?

我认为作为产品构建者，你应该思考如何真正利用这些轨迹和现实世界中的经验，并开始自主地塑造和改进产品。

I think as a product builder you should be thinking about how you actually take these traces and experience in the real world and start to shape and improve the product autonomously.

所以我们从模型开始，对吧，对于轨迹来说，我认为那里有一些有趣的工作要做。最终那将是模型和框架，基本上自我改进两者。我认为一旦你有了框架，产品就变得更加 AI 原生，所以框架在某种程度上成为许多公司产品的大部分，对吧，如果你有权自我改进框架，然后产品，那么你就有权改进并自我改进整个公司。

So we're starting with the model right for trajectory and I think there's some interesting work to do there. Eventually that will be the model and the harness and basically self-improving both of those. And I think once you have the harness and products are kind of becoming more AI native and so the harness is kind of becoming most of the product for a lot of these companies right well and if you have the right to kind of self-improve the harness and then the product then you have the right to kind of improve and and self kind of self-improve the entire company.

我认为那时产品和产品构建者开始变得几乎像园丁一样，就像你有一株植物，它会自主生长，而你作为产品构建者的目标基本上就是在地里设置这些棚架，指明我想在这里塑造我的产品。

And I think there then products and product builders start to become almost like a gardener, like you have your plant and it's going to grow autonomously on its own and your goal as a product builder is to basically just set these trellises in the ground of here's where I want to shape my product.

我认为这将是一个非常激动人心的世界，你可以直接把产品投放给几百个用户。它会自行成长，现在基本上你只需将你的品味注入公司成长的方向。所以我认为这将是一个非常激动人心的世界，公司应该开始朝着这个方向建设。

And I think this is going to be a very exciting world where you can just put a product out to a couple hundred users. It's going to grow on its own and now we're going to have basically you just imbue your taste into where the company grows. So I think that's going to be a very exciting world and companies should kind of start building towards that.

是的。对于正在听、心想“哇，Trajectory 听起来是个很酷的工作地点”的人，你的推介是什么？比如为什么有人会想加入 Trajectory 工作？

Yeah. For somebody listening who's thinking, "Boy, Trajectory sounds like a cool place to work." What's the pitch? Like why might somebody want to work at Trajectory?

所以，我认为有几件事。第一是持续学习作为一种范式是下一个学习范式。你能想到的上一时代的任何大公司。比如谷歌、亚马逊或 Facebook。它们基本上都在做持续学习。如果你想想推荐系统或广告，想法是从现实世界中获取所有这些偏好，然后创建在线系统，它们越用越能学习和变得更智能。

So, I think there's a couple things. The first thing is continual learning as a paradigm is the next learning paradigm. Any massive company that you can think of of the previous age. So, let's take Google or Amazon or Facebook. They were all doing continual learning basically. If you think about like recommender systems or ads, the idea is taking all of these preferences from the real world and then creating online systems that can learn and get smarter the more they're used.

我认为对于智能体来说，这还不存在，而这正是 Trajectory 想要占据的位置。我们从产品的 V1 开始，这是一个非常简单的训练模型的方式。但总的来说，我希望它成为一个庞大的推荐系统，每个智能体都应该适应公司的需求以及每个用户最终的需求，我认为这将是一个从根本上令人兴奋的世界。

And I think for agents that hasn't existed yet and that's exactly the place that trajectory I want to be in. We're starting with just V1 of the product which is a very easy way to train models. But in aggregate, I want it to be a massive recommender system of every single agent should be attuned to what companies want and what every single user eventually wants and I think that's going to be a fundamentally exciting world to get to.

有很多工作要做，我也认为 Trajectory 不是生态系统中唯一需要运作良好的公司才能实现这一点。你需要，比如在我们办公室两个街区外是 Modal，它做沙盒，为我们很多基础设施提供动力，再往两个街区是 Thinking Machines，也在旧金山，我认为所有这些类型的公司都需要被建立，基本上为所有基础设施提供动力，使持续学习发挥作用，但我认为产品界面是许多激动人心的工作可以发生的地方。

There's a lot of work that needs to happen and I also think trajectory is not the only company in the ecosystem that needs to work really well to make this happen as well. You need like two blocks down from us is Modal at our office that works on sandboxes and that powers a lot of the infrastructure that we have and two blocks down from that is Thinking Machines also in SF and I think there's like all these sort of companies that need to be built to basically power all the infrastructure to make continual learning work but I think the product interface is where a lot of the exciting work can happen.

是的。当我们几个月前第一次认识你时，你提到的一个想法是，现在这个世界拥有丰富的智能，这要归功于基础模型实验室，以及丰富的经验，这要归功于每天在许多不同领域辛勤工作的人们。但两者之间存在巨大的鸿沟。在某种程度上，持续学习正是弥合这一鸿沟的东西。它也是尊重人们所有来之不易的经验，并使其对基础模型产生的智能可读的东西。我认为这是一种非常优雅和美丽的方式来思考你们正在做的事情。然后由此引出的问题是，对于持续学习能带我们去向何方，你最乐观的看法是什么？

Yeah. One of the things you said when we were first getting to know you many many months ago was this idea that the world right now has plentiful intelligence thanks to the foundation model labs and plentiful experience thanks to all the people who do hard work in lots of different domains every day. But there's this big gap between the two. And in some ways, continual learning is kind of the thing that bridges that gap. And it's kind of the thing that honors all of this earned experience that people have and like makes it legible to the intelligence coming out of the foundation models. And I thought that was like a very kind of elegant and beautiful way to think about what it is you guys are doing. And then the question that that leads to is what is your most optimistic view of the future in terms of where continual learning can take us?

我喜欢这个问题。一个很好的例子是亚当·斯密的《国富论》。我们从 18 世纪开始，在这个世界里，每个人都在做同样的事情，对吧？你会做一个工匠的端到端工作流程，你可以这么称呼它。他基本上理论化的是，一个富足的世界需要专业化。你希望某人基本上不要一直切换上下文，而是专注于他们已经积累经验的事情。

I love that question. A good example of this is Adam Smith's Wealth of Nations. Where we started out in the 1700s in this world where everyone was kind of doing the same thing, right? You would do this end-to-end workflow, you could call it, of an artisan. And what he theorized basically is a world of abundance needs specialization. Where you want someone to basically not be context switching all of the time and work on the thing that they've built up experience about.

而在 AI 领域，现在似乎我们处于这个前工业革命社会。甚至是狩猎采集社会，每个人都在一遍又一遍地做同样的工作。我们在咨询一个技术神谕。我认为需要发生的是基本上进入一个不再只是一个汽车制造商、一个每个单元制造商的世界。

And it seems like in AI right now we're in this pre-industrial revolution society. Or even a hunter gatherer sort of society where everyone is just doing the same work over and over again. We're consulting one oracle of technology. I think what needs to happen is basically get to a world where it's no longer just one manufacturer of a car, one manufacturer of like every single unit.

但我们会走向一个专业化的世界，每个人基本上都能定制 harness、模型的每一个部分，那时我们就进入了一个丰裕的世界。企业里只有 1% 真正采用了 AI，这是有原因的。外面有太多工作流极其依赖人工，本可以自动化，并且借助这些可以投入生产的智能体，真正提升智能水平。所以我的目标基本上就是把这 1% 变成 99%，让我们真正从当下世界上积累起来的全部经验中学习。

But we get to a world of specialization where everyone can basically customize every single part of the harness, the model, and that's where we get to a world of abundance. And there's a reason why only 1% of enterprises have actually adopted AI. There are so many workflows out there that are super manual and could be automated, and actually increase intelligence based on all of these agents that could be in production. And so my goal is to basically take that 1% and turn that into 99%, so that we are actually learning from all of this built-up experience that lives in the world right now.

太棒了。感谢你来参加节目。

Love it. Thanks for being here.

谢谢，非常感谢邀请我。

Yeah, thanks so much for having me.