推理市场的未来与模型路由：商品化还是价值攀升？

我觉得我们把工程薪资的 30% 花在了编程工具上。过去两个月这个比例其实下降了，而我们用掉的 token 数量反而上升了。今天我们请到了 Harrison Chase，LangChain 的创始人，一家估值数十亿美元的公司，在创建和监控智能体方面处于领先地位。今天我们请他坐上热座，聊聊为什么今天构建智能体这么难，聊聊关于智能体的未来那些没人知道但人人都该知道的事，然后是我最喜欢的环节——一些别人都不信、但他确信一定会发生的古怪预测。

I think we spend 30% of our engineering salary on coding tools. It's actually gone down the last 2 months. While tokens, the number of tokens that we've used has actually gone up. Today we welcome Harrison Chase, the founder of LangChain, a multi-billion dollar company. It is a leader in the ability to create and monitor your agents. And today we welcome him to the hot seat to discuss why it is so hard to build agents today, what no one knows that everyone should know about what is coming with agents, and then my favorite, some really kooky predictions which no one else believes will happen, but he knows will definitely happen.

我真的低估了推理的需求。一切都跑得通。这个市场太大了，需求就是这么多。

I really underestimated the demand for inference. Everything just works. This market is so big. There's just so much demand for it.

准备好了吗，Harrison。老兄，我太期待这次了。我已经期待很久了，因为我记得 Miles 在 Benchmark 的时候跟我提起过你，对你赞不绝口。所以一直想促成这次对话。谢谢你来做客。

Ready to go, Harrison. Dude, I'm so looking forward to this. I've been looking forward to it for a while because I remember Miles telling me about you when he was at Benchmark and just singing your praises. So, wanted to make it happen for a while. So, thank you for joining me.

我很高兴能来。

I'm excited to be here.

那么，给不了解的人用 60 秒讲讲，LangChain 是什么，你们在技术栈里处于什么位置？

Now, for those that don't know, 60 seconds, what is LangChain and where do you sit in the stack?

我们构建工具，让构建智能体变得容易。所以我们主要做开发者工具。我们有一个开源框架，基本上把模型和你构建有用智能体所需的其他一切连接起来，比如工具、数据、计算等等。然后我们还有 LangSmith。LangSmith 是可观测性和评估。智能体总是会出错。你怎么知道它们是怎么出错的？你怎么衡量它们，从而随着时间提升它们的表现？这就是 LangSmith 做的事。

We build tools to make it easy to build agents. So, we build developer tools primarily. We have an open source framework which basically connects the model to everything else that you need to build a useful agent. So tools, data, computation, things like that. And then we also have LangSmith. LangSmith is observability and evals. Agents go wrong all the time. How do you know how they go wrong? And how do you measure them so that you can increase their performance over time? That's what LangSmith does.

好。我靠问蠢问题谋生。嗯，这就是我当播客的原因。为什么构建智能体这么难？

Okay. I ask stupid questions for a living. Um, that's why I'm a podcaster. Why is it difficult to build agents?

构建智能体难的地方在于，它们不按你希望的方式行事。模型是这些非确定性的黑箱，对吧？它们以自然语言为输入，然后输出自然语言。而且在黑箱内部，它们也是非确定性的。这意味着你得到的输入维度几乎是无限的，可以是任何东西。你把聊天机器人放在人面前，他们想打什么就打什么，对吧？这就让你很难知道他们会打什么。然后一旦进入模型——顺便说一句，智能体不只是一次模型调用，它们是一系列模型调用，还在和其他工具交互。所以你有这些小型的黑箱，被放进这个大系统里，基本上就会得到级联式的错误。你有非常大的输入空间，也有非常大的输出空间，因为两个相似的输入进去，可能得出完全不同的答案。让这一切变难的就是 LLM 本身。没错，它们非常非常聪明，但你不知道它们到底会做什么，而且输入稍有变化它们就会改变答案。所以你构建了这个系统，但它真的很不可靠。让这个系统可靠地运行，这就是最难的部分。当我们谈三四年前的智能体和今天的智能体，它们已经好多了。主要发生的事，一是模型变好了，二是我们搞清楚了如何构建能更好地驾驭这些模型的系统。而第二部分，也就是驾驭这些模型，正是我们专注的。

The hard part about building agents is they don't behave how you want them to behave. So, the models are these non-deterministic black boxes, right? And they take natural language as input and then they output natural language. And inside the black box, they're also non-deterministic. And so what that means is that the input you get is like infinite in dimension. It can be anything. You put a chatbot in front of someone, they can type anything that they want, right? And so that makes it really hard to know what they're going to type. And then once it goes into this model, and by the way, agents are not just one model call. They're a series of model calls and they're interacting with other tools. And so you have these small black boxes that are then put inside this large system and you get basically these cascading types of errors. So you have this very big input space. You have this very big output space because two similar inputs can go in and they can arrive at completely different answers. And the thing that's making all this hard is the LLM itself. Yes, they're very, very smart, but you don't know exactly what they're going to do, and they change answers based on small changes to the input. And so you build this system, but it's really non-reliable. And so getting this system to behave reliably, that is the hardest part. When we talk about agents 3 years ago, 4 years ago to agents today, they've gotten a lot better. And the main thing that's happened is one, the models have gotten better, and two, we've figured out how to build systems that basically harness these models better. And that second part, the harnessing of these models, is what we focus on.

随着模型变好，你们的业务会以哪些人们看不到的方式发生变化？

How does your business change in a way that people don't see as the models get better?

随着模型变好，我们业务变化的主要方式是我们基本上必须抹掉产品的一大部分。我的意思是——Cursor 也说得很好。我想这其实是他们公司的一条实际运营原则，叫“抹掉产品”或者“删掉产品”，我记得他们是这么叫的——基本上就是模型变好了，你 6 个月前必须做的事现在不用做了。这在我们开源项目里非常相关。开源基本上就是驾驭这些模型、帮它们做事的框架和工具。而驾驭这些模型、帮它们做事的正确方式每个月、每 3 个月、每 6 个月就变一次，随你挑时间跨度，但变化相当快。

The main way our business changes as the model gets better is we basically have to erase big parts of our product. So what I mean by that — and Cursor said this really well as well. I think this was one of their actual operating principles as a company, like erase the product, or delete the product, I think is what they called it — where basically these models get better and things that you had to do 6 months ago you no longer have to do. And so this is very relevant in our open source. So open source is all about basically the harness and the framework for taking these models and helping them do things. And the right way to take these models and help them do things changes every month, 3 months, 6 months, choose your time period, but it changes pretty rapidly.

面对如此不确定的未来，你怎么构建？你是不是只能和实验室绑在一起、了解他们的路线图，这样才不会花 6 个月做的东西被他们抹掉？

How do you build for such an uncertain future? Do you just have to get in bed with the labs to know the road map so you don't waste 6 months building something that then they erase?

没有简单的答案。一方面，你打算怎么办？不构建了吗？我喜欢对问这个问题的创始人这么说：我该怎么办，东西在变，模型在变，我现在做的东西 6 个月后可能就没用了。那替代方案是什么？你干脆什么都不做？所以我觉得你必须对未来哪些部分会保留、哪些不会保留做出有根据的猜测。比如早期我们在 LangChain 里做工具调用的方式是让模型生成 JSON DSL，然后解析它。当时我们所有人都觉得或知道这有点像个临时 hack，但我们必须这么做，因为那是唯一能真正引导它的办法，因为你得让它完成这种工具调用。然后几个月后，OpenAI 推出了工具调用。我想我们很早就意识到——这是工具调用更原生的版本，直接内置于模型和 API 中。我想我们很早就意识到那是未来。所以我们投入了大量精力，完全从 JSON 解析切换到这种工具调用。所以我觉得你基本上必须分辨哪些是短期 hack，哪些是长期的、会一直存在的东西。模型永远不会知道你的专有数据，它们永远无法真正拿到 API 的输出。你必须把它们的 API 调用传给 API，拿到输出，然后继续循环。所以我认为模型周围的系统、把它连接到其他地方，会一直存在。但我们围绕思维链、规划或工具调用做的很多 hack，那些可以更紧密地集成进模型或模型 API 的东西，会消失。我觉得你只需要快速识别这一点，然后继续前进。

There's no easy answer. On one hand, what are you going to do? Not build? That's what I like to say to founders who ask this, like, what do I do, stuff is changing, the models are changing, if I build something now it might not be relevant in 6 months. What's the alternative? You just don't build anything? And so I think you have to make educated guesses as to what parts are going to be around and what parts aren't going to be around in the future. So for example, early on the way that we would do tool calling in LangChain was by having the model generate JSON DSL and then parsing it. And that was something that I think we all thought or knew was a little bit of a hack at the time, but we had to do it because that was the only way to get it to actually steer things, because you have to get it to do this type of tool calling. And then a few months later, OpenAI came out with tool calling. And I think we recognized pretty early — and this is a more native version of tool calling that's directly in the model and in the APIs. And I think we recognized pretty early on that that was the future. And so we invested a lot of effort and just completely switched from JSON parsing to this tool calling. And so I think you have to basically recognize what things are short-term hacks and what things are long-term, will always be there. The models will never know your proprietary data. They will never be able to actually get the output of APIs. You have to take their API call, pass it to the API, get the output and then continue in the loop. And so I think the system around the model, connecting it to other places, will always be there. But a lot of the hacks we do around chain of thought or planning or tool calling, things that can just be much more closely integrated into the model or into the model API, will go away. And I think you just have to be quick to recognize that and go on.

为什么我们需要中立的驾驭框架？我们看到，说白了，模型公司自己就有非常出色的智能体构建者。

Why do we need neutral harnesses? We see, you know, bluntly very brilliant agent builders from the models themselves.

为什么你们这一层不会被它们吞掉？

Why does your layer not just get eaten by them?

我认为我们需要中立框架的原因是，在构建智能体之前或智能体运行过程中，你会积累大量状态。状态可以包括记忆，无论是短期记忆还是长期记忆。记忆也有不同的形式。它可以来自与智能体的交互，也可以来自智能体从环境中学习。状态之所以会积累，是因为智能体在交互。它出去、学习、聊天，并学习一些东西来让自己变得更好。我认为现在最有趣的事情之一就是如何让这些智能体随着时间的推移变得更好并学习。其中一些学习和随时间变好会体现在模型权重中，但很多是在模型之外。很多是在这个上下文中。现在当你构建这个上下文时，你真的不希望它被绑定到单一的模型提供商。原因在于，那样它就被绑定在那里了。你无法转向其他模型。能够更换模型是非常具有进攻性和防御性的。进攻性在于，无论最佳模型来自谁，当它出现时你都想利用它。防御性在于，你不想被锁定，这样他们就不能提高费率等等。所以你要能够切换模型。框架可以与模型紧密绑定，也可以是中立的。现在上下文与框架紧密相连。所以框架的工作基本上是拿着模型并与所有其他东西交互。所有其他东西包括数据和 API，但也包括记忆。所以如果你构建这个记忆的方式是将其与框架绑定，而框架又与模型绑定，那么你就有点被困住了。

I think the reason we need neutral harnesses is that there is a lot of state that you build up either before you build the agent or as part of the agent running. And the state can include things like memory, whether that's short-term memory or long-term memory. Memory also comes in different form factors. It can be from interacting with the agent versus from the agent learning from the environment. And the reason that this state gets built up is the agent interacts. It goes out, it learns, it chats and it learns things to make it better. I think one of the most interesting things right now is how to make these agents do better over time and learn. And some of that learning and some of the doing better over time will be in the model weights, but a lot of it is outside the model. And a lot of it is in this context. And right now when you build up this context, you really don't want it to be linked to a single model provider. And the reason for that is that then it becomes linked there. You can't go to other models. Being able to change models is very offensive and defensive. So it's offensive in the fact that you want to be able to take advantage of the best model when it comes out no matter who it's coming from. It's defensive in the fact that you don't want to get locked in so they can't raise rates, things like that. So you want to be able to switch models. The harness can be very tied to the model. It can also be neutral. The context right now is really intricately linked to the harness. So the harness's job is to basically take the model and interact with all the other things. All the other things includes data and APIs, but also includes memory. And so if you build this memory in a way that is linked to the harness and the harness is linked to the model, then you're kind of stuck.

抱歉，框架结构真正重要的是它允许用户的数据可移植性。

True of the harness structure, I'm sorry, is the portability of data that it allows the user.

是的。是的。你基本上不想被锁定在模型中。如果记忆、如果上下文开始以特定于某个模型希望如何与记忆交互或希望如何与上下文交互的方式积累，那对可移植性非常不利。然后这会使你在未来构建业务时处于劣势。

Yeah. Yeah. You want to basically not be locked in to models. And if memory, if context starts building up in a way that is specific to how one model wants to interact with memory or wants to interact with context, that's very bad for portability. And then that puts you at a disadvantage building your business going forward.

我们与很多人讨论过拥有自己的智能意味着什么，对此有很多不同的版本。我们认为拥有自己的智能意味着——我们认为有几个不同的组成部分，但一个关键组成部分是拥有这个开放且可控的智能体。就像你希望能够调整所有不同的部分。你希望能够调整模型。你也希望能够调整框架，因为有时这些模型实验室正在训练框架，使其非常擅长于它们分布内的事情，这通常是编程。编程很棒，有很多围绕编程的用例，但也有很多不是编程的用例。也许他们会写代码作为其中一部分，但他们将其用作销售、研究、法律以及这些其他垂直特定事情的方式。所以当你超出模型实验室训练分布时，你希望控制框架做什么。然后你也希望控制上下文。所以模型、框架本身和上下文构成了这个开放且可控的智能体架构。我们认为这是拥有自己的智能的重要部分。

We talk with a lot of people about owning their intelligence and what that means and there's a lot of different versions of that. The version that we think owning your intelligence means—we think there's a few different components of that, but one key component is having this open and controllable agent. Like you want to be able to adjust all the different pieces. You want to be able to adjust the model. You want to be able to adjust the harness as well because sometimes these model labs are training harnesses to be really good at the things that are in distribution for them, which oftentimes is coding. And coding is great and there's a ton of use cases around coding, but there are a ton of use cases that aren't coding. And maybe they'll write code as part of it, but they're using that as a way to do sales, as a way to do research, as a way to do law, as a way to do these other vertical specific things. And so when you get out of distribution of what the model labs are training on, you want to have control over what the harness does. And then you also want control over the context as well. So model, the harness itself, and the context make up this open and controllable agent architecture. And we think that's a big part of what it means to own your intelligence.

我想稍微回顾一下，当我们谈论智能体价值时。有人说价值实际上在于验证，比如它是否做得好，因为这解锁了下一步，即自动化。如果价值通过验证获得，谁赢？

I just want to back up a little bit when we think about agent value so to speak. Some people say that the value is actually in the verification like whether it did well or not because that unlocks the next step which is automation. If value is derived through verification, who wins?

框架。

The harness.

如果价值通过验证获得，我认为赢家是领域专家。那些能够量化价值含义的人，这非常难。比如构建评估是一项非常难的工作。所以我认为很多价值将归于那些能够编写验证器、捕捉领域知识的人。出于这个原因，我非常看好不同领域的垂直智能体，因为我认为最适合编写这些验证器或构建这些评估的领域专家，他们将能够构建框架和模型，不仅仅是框架，不仅仅是模型，而是它们的某种组合。所以非常实际地说，如果你开始构建垂直智能体，我强烈建议你从框架层开始，做上下文工程，调整框架，然后当你有了用户，再在后训练中调整模型。我认为从上下文或框架层开始回报率更高。所以我认为我们将拥有所有这些不同的垂直领域。他们能够以非常领域特定的方式量化对他们来说什么是好的，这将让他们不仅构建垂直智能体，还能构建垂直智能体体验。不同的智能体可能有不同的体验，所以我认为很多价值将——我非常看好垂直智能体，所以我们希望支持很多这样的东西。

If value is derived through verification, I think the people who win are the domain experts. The people who are able to quantify what that value means, which is very hard. Like building evals is a very hard job. And so I think a lot of the value will accrue to whoever can write those verifiers, capture that domain knowledge. I'm massively bullish on vertical agents for different domains for this reason because I think the domain experts who are best suited to write those verifiers or build those evals as we kind of talk about it, they will be able to build harnesses and models and it won't be just the harness, it won't be just the model, it will be some combination of them. So very practically if you're getting started building a vertical agent, I would highly suggest you start at the harness layer, you do context engineering, you tweak the harness and then when you've got users, then you start tweaking the model in post-training. I think it's much higher ROI to start at the context or harness layer first. And so I think we'll have all these different verticals. They'll be able to quantify what good looks like to them in a very domain specific way and that'll let them build not only vertical agents but also vertical agent experiences. Different agents might have different experiences and so I think a lot of the value will—I'm massively bullish on vertical agents and so we want to enable a lot of that.

你说如果我在构建那个，我会从框架层开始。我们看到很多，无论是 Harvey 还是我认为是 RAMP，都在构建自己的模型。他们实际上在多大程度上构建自己的模型？Mike Cannon-Brookes 在我们今天发布的上一期节目中表示，这很大程度上是尊重地说是胡说，他们是在微调开源的中国模型。这是真的吗？你如何看待？

You said like if I was building that I would start at the harness layer. We're seeing a lot whether it's Harvey or I think it's RAMP build their own models. To what extent are they actually building their own models? Mike Cannon-Brookes said the last on show we released today that it's largely kind of respectfully BS and they're fine-tuning open source Chinese models. Is that true? And how do you think about that?

我认为很多情况是他们在后训练开放权重模型。绝对如此。我仍然认为这是一件有价值的事情。我不认为这有什么错。我认为在拥有自己的智能方面，你可以有不同的维度。我认为最便宜的方式是你只控制上下文。你使用现成的框架、现成的模型。下一个层次是你开始调整框架，这比只控制上下文给你更多控制权，但做起来有点更贵。再下一个层次是你后训练一个开放权重模型。所以你可能拿一个现有的开放权重中国模型，为你的领域后训练它。然后最底层是他们实际上从头训练一个模型。

I think a lot of it is they are post-training open weight models. Absolutely. I still think that's a valuable thing to do. I don't think there's anything wrong with that. I think there's different dimensions that you can go in terms of owning your intelligence. I think the cheapest way is you just control the context. You use an off-the-shelf harness, off-the-shelf model. The next level down is you start tweaking the harness and that gives you more control than just the context, but it's a little bit more expensive to do. The next level down is you post-train an open weight model. And so you might take an existing open weight Chinese model and post-train it for your domain. And then the very bottom layer is they actually train a model from scratch.

我认为没有多少人在从头训练模型。大多数人是在做后训练。如果你在从头训练模型，那你可能是在某个新实验室。他们是少数在做这件事的人之一。但没错，像 Ramp 和 Harvey 这样的公司，他们拥有大量人才、大量数据、大量使用量，并试图真正优化它。他们已经从上下文中榨取了所有能榨取的价值，从 harness 中榨取了所有能榨取的价值，现在他们基本上已经到了对这些模型进行后训练的階段。

I don't think many folks are training a model from scratch. I think most are post-training. If you're training a model from scratch, you're probably at a neolab. Those are one of the few that are doing that. But yeah, the Ramps and the Harveys of the world, they have a ton of talent, a ton of data, a bunch of usage, and are trying to really optimize it. They've gotten all the juice they can out of the context, all the juice they can out of the harness, and now they've arrived at basically post-training of these models.

我能问你吗，关于这些 specialized 模型在后训练领域的有效性，目前还没有定论，就像我们提到的那样，我看到的一些报告说它们效率高得多、更好，而另一些则说没那么好。你认为这会如何发展？

Can I ask you, the jury's kind of out on the efficacy of these specialized models in the post-training realm, like we mentioned there, some reports I've seen say they're massively more efficient and better and some less so. How do you think this will play out?

我认为我们在后训练方面看到的是，目前实际有意义的很多后训练可能是针对构成更大 harness 的较简单任务。我的意思是，当你有一个 harness，一个在做有趣工作的智能体 harness 时，实际上很少只有一个模型在运行循环。那可能是核心，但还有很多其他模型调用，作为护栏、路由、子智能体或子工作流等等。所以我们现在看到后训练高效的很多地方，是针对较简单的任务，比如分类、提取或摘要，这些可能是智能体使用的工具或护栏之类的。我认为这类后训练现在非常合理，并且会继续下去。我认为更未回答的问题是，对于驱动循环的这个核心模型，因为智能体工作的主要方式是一个 LLM 在循环中调用工具。所以问题是，对于驱动那个循环的模型，那些会是后训练模型还是前沿模型？我认为这就是我们看到更多来回拉锯的地方，每次新的前沿模型出来，它们就更擅长运行这种核心循环。我们确实看到像 Harvey 和 Ramp 等公司在试验用后训练模型来驱动这个核心循环。所以我认为 Harvey 最近在过去一个月左右宣布了完全类似的东西。我认为这是一个开放的问题。

I think what we've seen in regards to post-training is that a lot of the post-training that practically makes sense right now is probably for simpler tasks that make up a larger harness. So what I mean by that is when you have a harness, when you have an agent harness that's doing interesting work, it's actually very rarely just one model running the loop. That might be the core of it, but there's a bunch of other model calls that happen either as guardrails or routing or subagents or subworkflows or things like that. And so a lot of the places where we see post-training being efficient now are for simpler tasks like classification or extraction or summarization that might be a tool the agent uses or a guardrail or things like that. I think those types of post-training right now make a ton of sense and will continue going forward. I think the question that is more unanswered is for this core model that's driving the loop, because the main way an agent works is there's an LLM running in a loop calling tools. So the question is for this model that's driving that loop, will those be kind of like post-trained models or will those be kind of like frontier models? And I think this is where we see more of a back and forth where every time a new frontier model comes out, the better they are at just running this kind of core loop. We do see people like Harvey and Ramp and others experimenting with post-training models for this core loop. And so I think Harvey recently in the past month or so announced something exactly in that vein. I think that's an open question.

如果你在构建自己的垂直应用，你会自己后训练吗？

If you were building your own vertical application, would you post-train your own or not?

一开始不会。绝对一开始不会，但如果我们谈论的是 5 年后，假设我获得大量使用量，我认为这可能是一个现实的可能性。

Not to start. Absolutely not to start, but if we're talking like 5 years in the future, assuming that I'm getting a ton of usage, I think it could be a realistic possibility.

我想 harness 还需要做的一件事是根据请求进行模型路由。模型路由层有多大价值？

One of the things that harnesses also need to do I imagine is model routing depending on request. How much value is in the model routing layer?

模型路由非常有趣。我认为这是目前最热门、也许最具争议的话题之一。所以有几点想法。首先，有一个问题是模型路由应该在哪里。应该在 harness 层还是在网关层？这有点进入技术细节了——

Model routing is so interesting. I think it's one of the hottest and maybe most divisive topics right now. So a few thoughts. First, there's a question of where the model routing should even be. Should it be at the harness level or should it be in the gateway level? And this is getting a little bit into the technical—

我们——是有区别的。抱歉。区别在于 harness 层是在运行代码中，而在网关层只是 API 调用。

We—there's a difference. I'm sorry. The difference is in the harness level it's in the code of how it runs and in the gateway it's just an API call.

所以把它放在网关层的好处是：因为它只是 API 调用，你可以把它放入任何 harness。你可以把网关层发生的路由放入 Claude Code，放入任何 harness，而无需更改代码。缺点是，也是为什么我更看好路由发生在 harness 中——我认为 Factory 和 Cognition 也写过相关文章——是因为在 harness 中，你对正在发生的事情有更多了解。你对状态有更多感知。当你有一个 API 时，它通常假设是类似 OpenAI 聊天补全 API，所以它只接收一定量的状态。你在 harness 中有更多状态来知道，嘿，提示缓存怎么样了。如果你想保持提示缓存热且可用，你也可以更容易地更改提示。所以很多这些模型有不同的提示策略。因此，如果你在 harness 层进行路由，你可以更容易地更改提示。所以我认为我们看到的很多是，第一，网关和 harness 之间的争论以及路由该去哪里。我认为我们可能相信它属于——

So the benefits of putting it in the gateway level: because it's just an API call, you can drop it into any harness. You could put this routing that's happening at the gateway level into Claude Code, into any harness without having to change the code. The downside, and why I think we're more bullish on routing happening in the harness—as are I think Factory and Cognition have also written things about this—is that in the harness you just have more knowledge of what's going on. You have more awareness of the state. When you have an API, it's generally assuming that it's kind of like an OpenAI chat completions API, and so it only takes in a certain amount of state. You have more state in the harness to know, hey, what's going on with the prompt cache. If you want to try to keep the prompt cache warm and available, you can also change the prompt more easily. So a lot of these models have different prompting strategies. And so if you do the routing at the harness layer, you can more easily change the prompt. And so I think a lot of what we've seen is one, this debate between gateway and harness and where does routing go. I think we probably believe it belongs in the—

为了让我们理解，Open Router 和所有其他提供商，有几个——

And just so we understand, the Open Routers and all the other providers, which there are several—

他们在网关层。

They are in the gateway.

他们在网关层。很多——我显然 Open Router 是不可思议的公司，不可思议的使用量——

They are in the gateway. A lot of—I obviously Open Router incredible company incredible usage—

请他们上过节目。喜欢 Alex。Alex 很棒。我认为人们使用 Open Router 和 Vercel 及其他网关的很多原因主要是为了模型访问。所以我认为路由器和网关之间也有区别。所以我可能会说 Open Router 的大部分使用量——Alex 显然比我更了解数字——但如果我必须猜，我会猜很多只是来自作为网关，而不是路由器。所以人们预先指定他们想要的模型，Open Router 只是提供一个单一的 API 和单一的 API 密钥来访问它。就智能路由而言，我认为实际做的比人们想象的少得多。而且我认为当它被做时，是像 Factory 这样的人构建到 harness 中的。我认为 Factory 在这方面做得非常出色。他们写了一些技术博客，关于他们如何进行路由，写得非常好。但我认为现在，我相当强烈地认为路由——你必须对 harness 有感知。你必须知道任务是什么。你必须知道 harness 是什么样子。所以我不认为会有一种路由既适用于法律又适用于金融又适用于编程，并且都一样。它将非常特定于领域和 harness。

Had them on the show. Love Alex. Alex is great. I think a lot of why people use Open Router and Vercel and other gateways is largely for the model access. So I think there's also a difference between a router and a gateway. And so I would probably say most of Open Router's usage—Alex obviously knows the numbers a lot better than I do—but if I had to guess, I would guess a lot of it just comes from being a gateway, not a router. So people specify the model they want up front and Open Router just provides a single API with a single API key for accessing that. In terms of smart routing, I think that's actually being done way less than people think. And I think when it's done, it's built into the harnesses by folks like Factory. I think Factory does a tremendous job about this. They've written a number of technical blogs that are very well done in terms of how they do routing. But I think right now I pretty strongly believe that routing—you have to be harness aware. You have to know what the task is. You have to know what the harness looks like. So I don't think you'll have routing that works for law and works for finance and works for coding and have it all be the same. It's going to be very domain and very harness specific.

抱歉。当我听你说时，似乎毫无疑问路由应该在 harness 层，考虑到那里存储的上下文和记忆。当我有这样一个明显的反应时，我想理解为什么我愚蠢？支持它在网关层的理由是什么？

I'm sorry. When I listen to you, it seems so unwaveringly obvious that routing should be at the harness layer given the context and memory that's stored there. When I have such an obvious response, I like to understand why am I stupid? What is the bull case for it being at the gateway layer?

我认为当人们谈论网关层时，他们可能在谈论两件事。第一，它使移植变得容易得多。所以你可以把 Claude Code——

I think when people talk about it at the gateway layer, there's maybe two things that they're talking about. So, one, it makes it way easier to port. So you can take Claude Code—

人们喜欢 Claude Code，对吧？如果你告诉人们，嘿，你不能用 Claude Code，你必须用这个其他的 harness。嗯，外面还有其他好的 harness，但我们先假设你很喜欢 Claude Code，你想让他们继续用，但你想做一些类似路由到 DeepSeek 或开源模型之类的事情，那么唯一的方法就是在网关层做，因为你不控制 harness。顺便说一句，这是支持开源 harness 的另一个论据，因为你可以控制和修改它。嗯，然后第二点是，当人们谈论网关层的路由时，我认为他们常常隐含地假设有一个领域。所以你可以说有两个通用领域，你也许可以为之构建通用路由器。我实际上认为 OpenRouter 在某种程度上已经做到了这一点。你可以说编码。编码是最流行的领域。所有使用都在那里。让我们构建一个对编码非常有效的路由器。也许我们不会感知 harness，但我们会感知用例。所以我们知道不同的模型在编码中擅长不同的事情，我们可以把它放在网关。另一件事你可以说，嘿，我们有一个通用的聊天机器人，我们不是想成为法律领域的顶尖，也不是金融领域的顶尖，但我们只是要成为一个公司范围内的通用聊天机器人，我们可以足够清楚地知道，如果你说“嗨，你好吗？”，你不需要 Opus 来处理。你可以用一个便宜的模型。所以我认为当人们谈论网关层的路由时，他们隐含地假设你是在做通用聊天或编码，而不是领域特定的东西。一旦你进入领域特定的东西，我真的认为它必须在 harness 层。你不会为此使用现成的网关路由器。

People love Claude Code, right? If you tell people, hey, you can't use Claude Code, you have to use this other harness. Um, and there are other good harnesses out there, but let's pretend for a second that you love Claude Code and you want to let them keep on using that, but you want to do some semblance of routing to maybe like a DeepSeek or an open source model or something like that, then the only way you can do that is at the gateway layer because you don't control the harness. Side note, this is another argument for open source harnesses that you can control and modify. Um, and then the second thing is when people talk about routing at the gateway layer, I think they're oftentimes implicitly assuming that there's a domain. So there's kind of like two general domains that you could maybe argue that you can build general routers for. And I actually think OpenRouter has done this to some extent. You could argue coding. Coding is like the most popular domain. That's where all the uses going. Let's build a router that works really good for coding. Maybe we won't be harness aware, but we'll be use case aware. So we know different models are good for different things in coding and we can put it at the gateway. The other thing you could say is like hey we've got this like general chatbot like we're not trying to be best in the class at law or best in the class at finance but we're just going to be like a general chatbot across the company and like we can know well enough that if you say hi how are you doing? You don't need like Opus for that. You can use a cheap model. And so I think when people talk about routing at the gateway level, they're implicitly kind of assuming that you're doing either like general purpose chat or coding and you're not doing like a domain specific thing. As soon as you get into a domain specific thing, I really think it has to be at the harness layer. You're not going to use an off-the-shelf gateway router for that.

所以你认为在 2 到 3 年内，大多数路由会在 harness 内，因为上下文带来的好处，但对于非常横向的任务，它会放在网关。

So in 2 to 3 years time you think the majority is within the harness given the benefits that come from that context but for very horizontal task it will sit in gateway.

我认为完全正确。如果你在构建垂直智能体，我认为路由需要在 harness 中。如果你考虑的是通用横向任务，比如只是通用聊天或编码，它也许可以在网关中。

I think that's exactly right. If you're building a vertical agent I think routing needs to be in the harness. If you're thinking about a general purpose horizontal task like just general chat or coding it could maybe be in the gateway.

它是商品吗？就像每个人都会构建它吗？

Is it a commodity? Like does everyone just build it?

我不认为路由是商品。我确实认为人们会构建它，但我认为即使今天构建它也是相当困难的，我认为部分原因是模型变化太快。所以你可以说某些模型在不同方面更好，但然后一个新的 Opus 或一个新的 OpenAI 模型出来了，它在所有方面都更好，所以路由的价值就下降了。

I don't think routing is a commodity. I do think people will build it but I think building it is quite hard even today and I think part of the reason for that is the models just change so rapidly. So like you can argue that certain models are better at different things but then a new Opus or a new OpenAI model comes out and it's just better at everything and so then the value of routing kind of goes down.

你信任评估吗？

Do you trust evals?

我认为评估是有用的，但我不会完全信任它们。我的意思是，我认为很多学术评估，当很多模型出现时使用的那些，现在其实不那么有用了。我认为所有出现的模型都在那些评估上达到了基准最大化，所以你学不到很多。我认为你拥有的、为你的领域构建的内部评估绝对仍然有用。所以我会说，这是构建评估成为构建垂直特定智能体的重要部分的主要原因。我认为你必须有一些衡量事物的方法，而且我认为你通常可以信任那些评估。嗯，但即使如此，最重要的事情之一就是看数据。就像看数据，看看它在做什么。不要过于依赖单个数字。我认为这就是为什么你需要追踪。你想看看发生了什么。也许你的评估不完美。评估真的很难。你可以有一些感觉，但也许模型做得更好，只是以你以前不知道如何捕捉的不同方式。所以我确实认为，是的，我认为你可以信任垂直特定的基准，但你也必须进化它们，这是我们看到的主要事情之一，因为这些模型只是以不同的方式变得更好。

I think evals are useful but I wouldn't trust them fully. And so what I mean by that is I think a lot of the academic evals that are out there that are used when a lot of the models come out are not really that useful anymore. I think all the models that come out are kind of benchmark maxed against those evals and so you don't learn a lot. I think internal evals that you have and that you build for your domain absolutely can still be useful. And so I would I mean this is the main reason that building evals is such an important part of building a vertical specific agent. I think you have to have some way of measuring things and and I think you can generally trust those evals. Um but even then one of the most important things to do is just look at the data. Just like look at the data and see what it's doing. Don't rely too much on kind of like a single number. I think this is why you want to have traces. You want to look at what's going on. Maybe your evals aren't perfect. Evals are really hard. You can have some sense but maybe models are doing better but in just a different way that you didn't know how to capture before. And so I do think uh yes I think you can trust vertical specific benchmarks but you have to evolve those as well and that's one of the main things that we've seen because these models just get better in different ways.

我可以问你吗？你在那里提到了追踪。如果追踪是智能体工作的尾气，那么价值在哪里？是应用、可观测性层、模型层？在哪里？

Can I ask you? You mentioned traces there. If like traces are the like exhaust of agent work, where is the value then? Is it the app, the observability layer, the model layer? Where is it?

如果你在构建智能体，我认为价值在于让那个智能体擅长你所在的领域或垂直领域的东西。我的意思是，再次，我认为有不同的方式控制智能体的行为。有上下文、有 harness、有模型。对其中任何一个拥有特定的专有东西都可能是一个优势。我认为控制上下文是最容易的，也是开始时最明显的事情，然后是控制 harness。那将是数据、记忆、特定 API、你可以使用的特定工具。控制 harness 有点难，但仍然你可以把很多领域知识放在那里，关于应该做什么，包括路由和事情，然后模型是最难控制的，但可能是最高的天花板，让你迭代每一个的东西是评估，所以这些有大量的价值，但只有当你捕捉洞察并带回上下文或带回 harness 或带回模型时才有价值，而那个确切的过程可能看起来不同，所以后训练非常适合将这些来自追踪的洞察带回模型。为了将东西带回上下文或记忆，你可能想要在那些追踪上运行另一个智能体，然后让那个智能体输出类似记忆的学习。然后为了将这些追踪中的学习带回 harness，你可能想要在那些追踪上运行一个智能体并更新 harness 本身的代码。我们实际上有一个东西为人们做这个。它叫 Engine。它是 LangSmith 中的一个智能体，在追踪上运行并尝试更新代码。然后当你更新这些东西时，你需要知道它实际上更好。这就是评估派上用场的地方。

If you're building an agent, I think the value is at what makes that agent good at the domain or vertical that you're working in. So what I mean by that is again I think there's different ways to control what an agent does. There's the context, there's the harness, and there's the model. Having specific proprietary things for any of those can be an advantage. I think controlling the context is the easiest and the most obvious thing to do when starting then controlling the harness. And that would be data, that would be memory, um that would be specific APIs, specific tools that you can use. Controlling the harness is like a little bit harder. uh but still you can still put a lot of your domain knowledge around what should do both routing and things in there and then the model is is the hardest control but potentially the highest ceiling the thing that lets you iterate on each of those is eval so there is a ton of value in these but there's only value in that you like capture insights that you bring back to the context or bring back to the harness or bring back to the model and that exact process can look different so post-training great for bringing these insights from traces back to the model. For bringing things back to the context or memory, you're probably going to want to run like another agent over those traces and then just and then have that agent output kind of like learnings for memory. And then for bringing the these learnings in the traces back to the harness, you're probably going to want to run an agent over those traces and update the code of the harness itself. And we actually have something that does this for people. It's called Engine. It's an agent in LangSmith that runs over traces and tries to update code. And then and then when you update these things, you're going to need to know that it's actually better. And this is where eval come in handy.

我可以问一下持续学习如何改变智能体及其工作方式吗？

Can I ask how does continual learning change agents and how they work?

持续学习，我不认为它会太多改变智能体的工作方式，但我认为它会改变你希望在智能体周围运行的系统。我的意思是，我认为持续学习智能体的核心循环，核心内循环也许说仍然大致相同。

Continual learning, I don't think so much changes how agents work, but I think it changes the systems that you want to have running around the agent. So what I mean by that is I think the core loop for a continual learning agent the core inner loop maybe to say will still largely be the same.

它将是一个在循环中运行、调用工具、与模型交互并在环境中运行的 LLM。我认为持续学习增加的是一个外循环。一个外循环，它观察智能体的输出，然后对其做处理。再次强调，它的具体做法可能因你对持续学习的定义而异。所以，我认为持续学习也是那种没有非常精确定义的词之一。或者也许我会把它定义为一个随着运行而随时间变好的系统。再次强调，它可以通过不同方式随时间变好。它可以更新上下文。它可以更新执行框架的代码，或者它可以更新模型的权重。而外循环对所有这些来说看起来大致相同。它会看起来像：获取轨迹，得到学习成果，修复一些东西，测量以确保它改进了，然后重复。但那种确切的方式，比如再次更新模型权重，与更新一个文本文件（比如记忆文件）非常不同。

It will be an LLM running in a loop calling tools, interacting with a model and in an environment. I think what continual learning adds is an outer loop. An outer loop that looks at the exhaust of agents and then does things with it. And again, what it does could be different depending on what continual learning means for you. So, I think continual learning is also one of those words that doesn't have a super precise definition. Or maybe I would define it as a system that gets better over time as it runs. And again, there's different ways it could get better over time. It could update the context. It could update the code of the harness or it could update the weights of the model. And the outer loop will look roughly the same for all of those. It will look like take traces, get learnings, fix something, measure it to make sure it improved and repeat. But that exact kind of like changing again updating weights of model very different from updating a text file that is like a memory file.

如果我们看上下文，就像我们提到的数据是其中一部分，我们看到数据公司以史上最快的速度扩展到数亿美元营收。我是说真的史上最快，而且每个人都在卖数据。鉴于你保持中立，不参与其中但与之相邻，你预计这个市场会如何发展？未来几年这会是一个整合的市场吗？你怎么看它如何发展？

If we look at the context like we mentioned data being a part of that, we're seeing data companies scale to hundreds of millions of revenue in the fastest time ever. I mean like ever ever and everyone is selling data. How do you expect this market to play out given your neutrality on not being involved in it, but being adjacent to it? Will this be a market of consolidation in years to come? How do you see it play out?

我实际上认为数据市场从大局来看仍处于早期，尤其是当我们谈论智能体的上下文层时。所以，我的意思是，有很多数据库，ClickHouse 可能是一个最近表现非常好的绝佳例子。我认为它们仍然主要专注于它们是一个数据库。它们是老式的数据库。它们擅长存储数据和查询数据，但仍然是以相当传统的方式。ClickHouse 在 AGI 之前就存在了，当我们谈论智能体的上下文层时。再次强调，会有大量数据从软件系统产生，我认为像现有的数据库 ClickHouse、Elastic 等等对此很擅长。当我们谈论智能体的上下文层时，我认为存在的一个关键部分是，是的，有一个数据库，但我实际上认为数据库周围有一个系统。我认为智能体的上下文层看起来更像数据库周围的智能体，而不仅仅是数据库。原因是，将所有这些学习成果摄取到数据库中，你需要某种智能来做到这一点。而我认为这种智能实际上非常非常难。记忆中最难的部分实际上根本不是数据库。当我们与许多公司交谈时，他们做记忆的方式就是把它存储在一个文件中。非常非常简单。难的部分是获取所有发生的观察，无论是来自用户与比如传统 SaaS 交互软件，还是用户与智能体交互，并将其提炼成足够有价值的信息，放入这个上下文层。这就是目前记忆中的难点。

I actually think the data market is still pretty early in the grand scheme of things in so much as we're talking about a context layer for agents. So, what I mean by that is like there are a bunch of databases that are ClickHouse is maybe a really good example of one that's doing incredibly well recently. I think they're still largely focused on they're a database. They're good old fashioned database. They're good at storing data and querying data, but it's still in pretty traditional ways. ClickHouse existed pre-AGI when we talk about a context layer for agents. And again like there will be a bunch of data produced from software systems and great like I think like existing databases ClickHouse Elastic whatever are great for that. When we talk about a context layer for agents I think a key part of what exists there is yes there's a database but I actually think there's a system around the database. I think like a context layer for agents will look more similar to like an agent around a database than just a database. And the reason for this is that taking all of these learnings and ingesting them into a database, you need some sort of intelligence to do that. And I think that intelligence is actually really really hard. The hardest part about memory is actually not the database at all. When we talked to a number of companies, the way that they do memory is they just store it in a single file. Very very simple. The hard part is taking all of the observations that happen whether it's from the user interacting with like let's say it's a traditional SaaS interacting with the software or the user interacting with the agent and distilling that into information that's valuable enough to go into this context layer. That's the hard part in memory right now.

为什么这么难？

Why is that so hard?

因为让 LLM 做你想让它做的事很棘手。而这种元反思，我认为是 LLM 实际上仍然很挣扎的事情，比如反思提示并改进提示，反思智能体架构并改进智能体架构。我们应该在内部为此建立一个基准。我们确实有一些基准，比如 engine，这是我们的智能体，正在运行一些东西。但我认为这种元反思对智能体来说仍然非常困难。我认为这对人类来说通常也是一件难事。就像我不知道我们是否真正理解，有些人可能理解，所以我不该这么说，但我不知道我们是否理解我们大脑内部发生了什么。我觉得我们的大脑仍然是那种复杂的东西，我们并不完全理解。所以如果我们现在试图在一个系统中复制这种学习过程，我认为有很多不同的部分。所以我认为你必须，是的，我认为模型不擅长反思。而且我认为我们不知道这个系统应该是什么样子。输入应该是什么样子？输出应该是什么？这个智能体的执行框架应该是什么？不同的工作流程应该是什么？

Because it's tricky to get LLMs to do what you want it to do. And this type of meta reflection I think is something that LLMs actually still struggle at a lot like reflecting on prompts and improving prompts, reflecting on agent architecture, improving agent architecture. We should build a benchmark for this internally. We do have some benchmarks for this in terms of engine which is our agent that's running on things. But I think this type of meta reflection is still really difficult for agents. I think it's also generally like a hard thing for humans to do as well. Like I don't know if we actually understand, some people probably do, so I shouldn't say this, but I don't know if we understand like what goes on inside our brain. Like I feel like our brains are still kind of complex things that we don't fully understand. And so if we're trying to now replicate this kind of learning process in a system, I think there's a bunch of different pieces. And so I think you have to I yeah, I think models aren't great at reflection. And I don't think we know what this system should look like. What should the inputs look like? What should the outputs be? What should the harness for this agent be? What should the different workflows be?

12 个月后模型在反思方面会有实质性的提升吗？

Will the models be meaningfully better at reflection in 12 months?

可能吧。我的意思是，它们似乎会在所有方面都变得更好。

Probably. I mean, it seems like they'll get better at everything.

你认为模型改进的速度会继续以同样的加速方式持续下去，还是你认为我们正在接近某种渐近的平台期？

Do you think the speed of model improvement will continue in the same manner of acceleration that it has done or do you think we are approaching some form of asymptotic plateauing?

我认为模型改进的速度，如果不身处实验室内部，可能真的很难说。我想说，我认为有很多任务，模型改进会趋于平缓，因为已经足够好了，就像我认为有一堆任务模型已经足够好了。

I think the speed of model improvement is something that's probably really tough to say without being inside of a lab. I would say I think there are a lot of tasks for which the model improvements will flatline in the sense that like it's already good enough like I think there are a bunch of tasks where the models are just good enough.

比如编程？

Which would be like coding?

是的，编程正在达到那个水平。比如摘要、路由任务。可能还有更困难的任务，或者我们甚至不知道去思考、去尝试让模型处理的任务。我可能会说，这种元学习就是模型现在不擅长的任务之一。我想象这对实验室来说非常重要，比如破解 RSI 之类的事情。所以我确实认为模型会随着时间的推移在这方面变得更好，而且进展实际上可能看起来比之前快得多。我认为如果不身处实验室内部，真的很难有有根据的观点。有些任务已经饱和。还有其他任务，我认为可能只是比以前增长得更快。

Yeah, coding's getting there. Like summarization, routing tasks. There are probably more difficult tasks and or tasks that we don't even know to think about, know to throw a model at. I would probably say like this type of meta-learning is one of those tasks that the models aren't good at now. I would imagine is very important to the labs in terms of cracking RSI or things like that. And so I do think the models will get better at that over time and it's possible that that progress actually looks much faster than previous. I think it's really tough to have an educated opinion without being inside of a lab. There are some tasks where it's saturated. There are other tasks where I think it will probably just increase faster than before.

今天听起来很疯狂，但你认为在 2 到 3 年内会变得非常普遍的智能体应用是什么？

What about agents today sounds crazy that you think will be incredibly common in 2 to 3 years?

我的意思是，我认为智能体的这个记忆层仍然很大程度上未解决。我认为智能体今天罕见但未来几年会变得非常普遍的一件事就是这个记忆层。

I mean, I think this memory layer for agents is still massively unsolved. I think one thing about agents that is rare today but will be incredibly common in the next few years is this memory layer.

所以，我的意思是，如果你看看 Instinct 或 Muse，我认为他们做对的一件事就是这种个性化、学习，以及让它感觉像是为你量身定制的，无论是通过语气，还是通过那种主动跟进。所以我认为记忆还非常早期。我们和人们讨论记忆已经大约两年了，并尝试在开源方面做一些事情。我们发布了一个 alpha 版本，然后砍掉了一个商业化的记忆解决方案。记忆还太早期，而且非常领域特定。对于 Instinct 或 Muse 来说，记忆的样子实际上与公司或 harness 可能的样子相当不同，也与法律或金融等领域的样子相当不同。所以我认为记忆是真正能带来差异的一件事。你看到这带来了差异——我认为 Instinct 和 Muse 在这方面做得很好，所以你看到了它如何在那里产生差异。但我认为它仍然非常早期，我预计几年后所有产品都会包含某种形式的记忆。

So, what I mean by that is if you look at like Instinct or Muse, I think one of the things that they've nailed is this kind of personalization and learning and seeming like it's something for you, whether it's because it's speaking in the tone or because it's just following up like that proactiveness. And so I think memory is super early. We've been talking with folks about memory for like 2 years and trying to do things in the open source. We launched an alpha and then killed a commercial solution for memory. It's just so early on in memory and it's so domain specific. What memory looks like for Instinct or Muse is actually pretty different than what it probably looks like for a company or harness and it's pretty different than what it looks like for legal or finance or things like that. So I think memory is one thing that really makes a difference. You see this make a difference—I think Instinct and Muse have done a great job at this and so you see how it makes a difference there. But I think it's still very early on and I would expect all products in a few years to have some aspect of memory in it.

恕我直言，这难道不是相当明显吗？

With respect, is that not quite obvious?

我认为记忆会产生影响这一事实是相当明显的，这就是为什么你看到这么多人对它感兴趣，并且在过去几年里一直感兴趣。我仍然认为答案并不明显。我仍然认为答案极其不明显。我不认为有人知道记忆的答案是什么。

I think the fact that memory will have an impact is quite obvious and that's why you see so many people interested in it and have been interested in it for the past few years. I still think it's non-obvious what the answer is. I still think it's incredibly non-obvious what the answer is. I don't think anyone knows what the answer for memory is.

既然它如此明显，而且几乎在所有东西中都有明确需求，你为什么砍掉了它的商业解决方案？

Why did you kill the commercial solution for it given the obviousness of it and the clear need for it in almost everything?

我们砍掉它是因为我们没有找到产品市场契合点。为什么我们没有找到产品市场契合点？我认为我们实际上试图做得太通用了。我认为记忆现在太早期了，许多你想构建的解决方案实际上非常领域或智能体特定。

We killed it because we didn't find product market fit. Why didn't we find product market fit? I think we actually tried to be too general. I think memory is so early right now that a lot of the solutions that you want to build are actually very domain or agent specific.

所以你会构建记忆——比如垂直特定的记忆解决方案。你会怎么做不同？

And so you would build memory—like vertically specific memory solutions. What would you do differently?

回到记忆的难点是什么。我认为记忆的难点在于从事实中学习该记住什么的过程。而那个进行记忆的智能体，正如我所说，我非常看好垂直智能体。我认为让这些智能体擅长学习其领域特定的事物是其中的关键部分。所以，我确实认为这些学习型智能体，至少一开始，可能必须是领域特定的。我不知道是否存在一个通用的——这些学习型智能体的 harness 是什么样子？它可能看起来和我们现在的许多 harness 非常相似。我们构建深度智能体。深度智能体是一个智能体 harness。可能看起来类似。一个 LLM 运行循环，但它有特定的指令和特定的工具，也许还有一些特定的子智能体，非常擅长导航现有的记忆结构或提炼事实。所以我认为你会有用于记忆事物的智能体。我只是认为它们会是领域特定的。

Going back to what the hard part of memory is. I think the hard part of memory is that process for learning from facts about what to remember. And that agent that does that remembering, as I said, I'm massively bullish on vertical agents. And I think having those agents be good at learning things specific to their domain is a key part of that. And so, I do think these learning agents, at least to start, probably have to be domain specific. And I don't know if there's a general—what does the harness for these learning agents look like? It probably looks pretty similar to a lot of the harnesses we have now. We build deep agents. Deep agents is an agent harness. Probably looks similar to that. An LLM running the loop, but it's got specific instructions and specific tools and maybe there's some specific sub-agents that are really good at navigating existing memory structures or distilling facts. And so I think you'll have agents for remembering things. I just think they'll be domain specific.

你认为会有面向消费者应用的垂直智能体吗？我理解并同意你的观点，当你看看金融、会计、销售、营销、法律。你提到了 Muse 和 Instinct。它们会是水平智能体。你认为我们会有垂直消费者智能体吗？

Do you think you will have vertical agents for consumer applications? I understand and I agree with you when you look at finance, accounting, sales, marketing, legal. You mentioned Muse and Instinct. They would be horizontal agents. Do you think we'll have vertical consumer agents?

我认为我对消费者领域的看法是两者都会有。你会有水平和垂直智能体。水平智能体将与垂直智能体交互。

I think my take on the consumer space is that you'll have both. You'll have both horizontal and vertical agents. And the horizontal agents will interact with the vertical agents.

在哪里有垂直智能体是有意义的，哪里没有？

Where does it make sense to have a vertical agent versus not?

我认为对于大多数应用来说，拥有一个垂直智能体可能是有意义的。我认为拥有垂直智能体有意义的理由可能是双重的。也许更多，但有两个跳入脑海。第一，许多这些应用拥有关于你的知识，并且能比一般的水平智能体更好地学习你如何与该应用交互。所以回到领域特定性的点，你如何与音乐交互，比如 Spotify，与你如何与 Strava 交互非常不同，对吧？所以那些应用实际上可能最擅长学习关于你的事情。我认为暴露这些学习的最佳方式可能是通过一个智能体。另一件事是许多这些应用有大量的 API。所以你可以说，嘿，Spotify，不要构建智能体，只需暴露一堆 API。我实际上认为如果有一个智能体包装那些 API 并暴露一个单一的——或者也许多个但可能是一个单一的——自然语言界面，让水平智能体与那个垂直智能体交互，体验会更好。

I think for most apps it probably makes sense to have a vertical agent. And I think the reason that having vertical agents makes sense at all is probably twofold. Maybe more, but two that jumped to mind. One, a lot of these apps have knowledge about you and can learn things about how you interact with that app better than a general horizontal agent could. So back to the point about domain specificity, how you interact with music for say like Spotify is very different than how you might interact with Strava, right? And so those apps are actually probably best at learning things about you. And I think the best way to expose those learnings is probably through an agent. The other thing is a lot of these apps have a ton of APIs. And so you could say, hey, Spotify, don't build an agent, just expose a bunch of APIs. I actually think it's a better experience if there is an agent that wraps those APIs and exposes a single or maybe multiple but probably a single natural language interface for a horizontal agent to interact with that vertical agent.

当智能体成为网站的基本接口时，基础设施如何变化？我的意思是，我想要某个特定音乐会的门票，我说每 5 分钟检查一次，并购买任何可用的座位，三天里他们每 5 分钟检查一次。查询量是人工情况下的千倍。在智能体交互或人类交互的世界中，所需的基础设施如何变化？

How does the infrastructure change when agents are the fundamental interface with sites? And what I mean by that is, I wanted tickets to a specific concert and I said check every 5 minutes and buy any seat that comes available and for 3 days they checked every single 5 minutes. The volume of queries was a thousand times what it would have been with humans. How does the infrastructure required change in a world of agent interaction or human interaction?

当智能体交互时，请求数量会大幅上升。我想你看到过一个故事，我认为是 Resy 封锁了一个用户的账户，因为他们使用 Instinct 来拉取——我不知道频率如何，但相当频繁。所以我认为是的，基础设施——基础设施承受的压力要大得多。

When agents are interacting it makes the number of requests go way up. I think you saw a story about how I think it was Resy that blocked a user's account because they were using Instinct to pull it—I don't know how frequently but pretty frequently. And so I think yeah, the infra—there's just way more strain on the infrastructure.

这如何渗透到更低层？

How does that percolate down to lower layers?

我还不确定我对此是否有看法。我认为你必须构建能够更加突发、能够扩展得更广的系统。我不知道我是否有更多具体细节。

I don't know if I have an opinion on that yet. I think you have to build systems that can be much more bursty that can scale way broader. I don't know if I have more specifics.

你认为提供商会允许智能体使用吗？

Do you think providers will allow agent usage or not?

我认为未来是提供商暴露自己的智能体，让人们要么与该智能体交互，要么让人们的水平智能体与该智能体交互。

I think the future is providers exposing their own agent and having people either interact with that agent or having people's horizontal agents interact with that agent.

所以我的水平智能体，抱歉，我理解这会与 Resy 的垂直智能体交互。

So my horizontal agent, sorry, I just understand this would interact with Resy's vertical agent.

我认为完全正确。

I think that's exactly right.

为什么会奏效？因为这仍然没有解决许多这些企业的两个核心广告问题。

Why would that work? Because that still doesn't solve the two core problems of advertising for a lot of these businesses.

他们失去了那个广告模式，然后追加销售，他们失去了你可能喜欢的其他东西，所以你的购物篮规模就缩小了。为什么智能体会起作用？我认为你可以把其中一些东西包含在智能体的回复中。所以你可以为 Resi 或 Etsy 等平台上的特定商品做广告，并将其包含在智能体的回复中，比如“嘿，我找到了这三样东西，你可能还想考虑这个”。现在这变成了一个自然语言的回复，返回到主智能体。我还认为，关于系统压力这一点：如果你有一个垂直智能体，它在与底层系统交互方面会比第三方智能体更好、更高效。所以我认为这会使他们这样做效率高得多。

They lose that ads model and then upsell, they lose the like anything else you might like also on the side, so you get reduced basket size. Why would agent work? I think you could include some of those things in the agent response. So you could do advertising for specific things on Resi or Etsy or whatever and include that in the agent's response, like hey I found these three things, you might also want to consider this other thing. That's now a natural language response that goes back to the main agent. I think also to the point around the strain on the systems: if you have a vertical agent, it's going to be better and more efficient at interacting with the underlying systems than a third party agent will. So I think it will make it a lot more efficient for them to do that.

当我们看到查询量和智能体带来的变化时，是的，完成所有这些工作需要大量 token。token 的成本和智能的成本随时间如何变化？因为我们一直以为它会变得更便宜。但并没有便宜多少。我们实际上该如何看待这个问题？

When we look at the amount of queries and the volume that changes with agents, yeah, it takes a lot of tokens to do all of this work. How does the cost of tokens and cost of that intelligence change over time? Cuz we always thought it would get cheaper. Hasn't got much cheaper. How do we actually think about that?

成本是我们现在与人们讨论的一大问题。我认为这有几个不同的维度。第一，前沿模型相对而言随时间变得更便宜。与一年前存在的许多东西相比，Astra 的性价比非常出色。第二，需求大大增加。当你开始谈论智能体与智能体对话或多智能体系统时，燃烧的 token 数量要多得多，所以成本在上升。我们主要在编程中看到这一点。编程成本飙升，我认为包括我们在内的许多组织现在都在思考如何削减这些成本。这最终如何解决？如果你看看我们如何处理编程智能体，我认为有一些线索。当我们处理编程智能体时，我们分三步考虑。第一，我们考虑的基本上就是可见性，向管理员显示东西将花费多少、已经花费了多少。第二是成本控制。所以你可以说，“嘿，去给我订个位子，但别花超过 5 美元。”然后第三是实际影响 harness 本身。我认为主要来说，很多 harness 工程都是围绕让智能体更高效进行的，因为那是——抱歉，很多 harness 工程都是为了让智能体更好、更准确，因为那是最大的障碍。我认为随着成本变得更受关注，我们看到人们更关心提示缓存之类的东西，这对降低成本非常有效。我们看到模型提供商实际上通过——这有点深入细节了——但他们引入了这样的功能：你可以在飞行中更改智能体可用的工具。以前，这会破坏缓存并导致成本大幅增加。现在，你可以动态添加这些并保持成本不变。所以，我认为我们看到在控制成本方面，人们对模型 API 和 harness 都更加关注。

Cost is one of the big things that we talk with people about right now. I think there's a few different dimensions to this. One, I think the frontier models are relatively getting cheaper over time. Astra is like price performance fantastic compared to a lot of things that existed a year ago. Two, there's just way more demand. When you start talking about agents talking to agents or multi-agent systems, there are just so many more tokens that are getting burned and so the costs are going up. We see this in coding primarily. Coding costs have skyrocketed and I think a lot of organizations including ours are now thinking about ways to kind of pair those costs back. How does this end up getting resolved? If you look at how we're tackling coding agents, I think there's a few hints there. When we're tackling coding agents we think of it in three steps. One we think of basically just like visibility, showing how much things will cost, how much things have cost to admins. Two is cost control. So you could say something like, "Hey, you know, go get me a reservation, but don't spend more than $5 doing it." And then third is actually impacting the harness itself. I think primarily a lot of harness engineering has been done around making the agent more efficient because that's the—sorry, a lot of the harness engineering has been done to make the agent just like better, more accurate because that's the biggest blocker. I think as cost becomes more concerned, we're seeing people care more about things like prompt caching which are really good for reducing costs. We're seeing model providers actually make this easier by—and this is getting a little bit in the weeds—but they introduced things like where you can change the tools that are available to an agent mid-flight. Previously, this would have busted the cache and caused a massive increase in cost. Now, you can just add those dynamically and keep the cost the same. So, I think we're seeing more attention to both the model APIs and harnesses in terms of controlling costs.

当你今天与大型企业交谈时，他们最关心什么？那么对他们来说最重要的是什么？是准确性？是成本还是安全性？

When you talk to large enterprises today, what do they matter most? So what matters most to them? Is it accuracy? Is it cost or is it security?

顺便说一句，另一个是延迟。所以我们每年进行开发者调查，询问开发者他们关心什么。过去两年，第一位的回答——基本上是接下来两项的两倍——是准确性。所以那仍然是首要的。接下来两项是延迟和成本，安全性稍微靠后一点，但很接近。当我们最近与大型企业交谈时，我会说成本在关注度上有所上升。我仍然可能说它低于准确性，但可能高于延迟。然后安全性——对于大型企业，安全性更高，但可能仍低于准确性和成本。

Another thing would be latency by the way. So we run developer surveys every year where we ask developers what they care about. For the last two years the number one response—basically double the next two—has been accuracy. So that's still the primary thing. The next two are latency and cost and security is like a little bit after that, pretty close. When we talk to large enterprises recently, I'd say cost has risen in terms of concern. I'd still probably say it's below accuracy, but it's probably higher than latency. And then security is—for large enterprises, security is higher, but still probably below accuracy and cost.

我能问一下你们工程薪资的大概百分之多少？我不是特指，但你们在编程工具上花费多少？

Can I ask you what percent of your engineering salary give or take? I'm not specifically, but do you spend on coding tools?

我认为我们在编程工具上的花费大约占工程薪资的 30%。实际上过去两个月有所下降。而 token，我们使用的 token 数量实际上上升了。所以我们通过三个步骤做到了这一点。第一基本上就是可见性。我们将所有编程智能体的使用情况跟踪到 Langsmith。所以我们可以知道谁花了多少，花在什么上。我们可以对此进行分析。第二就是基本的成本控制。我们设定了一个预算，如果你达到预算，你可以和我们的工程副总裁谈谈，他几乎总是会增加预算，但这只是防止智能体被错误使用或陷入不良循环的保障。第三是我们控制 harness。所以我们做的一件事是我们实际上构建了自己的内部云编程智能体。所以不是 IDE，不是基于终端的智能体。我们让人们随意使用那些，但我们构建了自己的基于云的智能体，我们称之为 Open Suite。实际上这是我唯一编码的方式了，因为我一直在 Slack 上，所以我只需从 Slack 发消息，它就会启动。但在那个 harness 中，我们做了大量工作来控制成本。所以像提示缓存，还有我们内置到这个 harness 中的路由之类的东西，我实际上认为这是一个相当可重复的三步流程，用于控制编程智能体的支出。首先是可见性，第二是基本控制，第三是尝试改进 harness 本身。

I think we spend probably roughly 30% of our engineering salary on coding tools. It's actually gone down the last 2 months. While tokens, the number of tokens that we've used has actually gone up. And so this has—we've done this by doing three steps. First is basically just visibility. We tracked all of our coding agent usage to Langsmith. So we could tell who was spending how much, what they were spending it on. We could do analytics over that. Second is just basic cost controls. We set a budget and if you hit the budget, you can talk to our VP of engineering and he'll almost always increase it, but it's just a safeguard against agents being used incorrectly or going in a bad loop. And then third would be we control the harness. So one of the things we've done is we actually built our own in-house Claude Coding agent. So not an IDE, not a terminal-based agent. We let people use whatever they want for that, but we built our own cloud-based agent, Open Suite is what we call it. It's actually the only way I code anymore because I'm in Slack all the time and so I just message it from Slack and it kicks off. But in that harness, we've done a bunch of work to control costs. So things like prompt caching, but also things like routing we've built into this harness and I actually think that's a pretty repeatable three-step process for getting coding agent spend under control. Just first visibility, second basic controls, third try to improve the harness itself.

完全理解。我只是想深入探讨一下。你提到可见性，看看谁在花什么，花钱最多的人是最有效的吗？

Totally get that. I do just want to unpack that. You mentioned that visibility, see who's spending what were the people spending the most the most effective.

有时花钱最多的人是最有效的，其他时候则不是。我的意思是，我们有几位高级工程师花费很多，我 100% 相信他们所做的是值得的。毫无疑问。我们也曾遇到过一些情况，更多是由于愚蠢的错误或设置不当，而不是其他原因，导致使用量大幅飙升。举个例子：我们有一两个任务，使用了一些插件，基本上会截图或获取网页 DOM 的快照，并在智能体的每一轮都发送回去，因为他们在做前端工程，他们想获取外观并发送回去。它就会运行这个巨大的循环。

Sometimes the people spending most are the most effective and other times they're not. So what I mean by that is we have a few senior engineers who spend a lot and I 100% trust that what they're doing is well worth it. No questions there. We've also just had some cases where due more to silly mistakes or improper setup than anything, we've had massive spikes in usage. So an example of this: we had one or two tasks where some plugin was being used that basically would screenshot or take a snapshot of the DOM of a web page and send that back every turn of the agent because they were doing some front end engineering and they wanted to take what it looked like and just send it back. It would just run this massive loop.

所以，我觉得我们在一周内就在那一件事上花了 4 万美元。所以，当事情发展到那种程度时，再次强调，我认为我们作为一家公司相当开明，并且信任员工，但我知道那可能不值得。所以，这只是一个例子，再次说明，这不是恶意的。甚至不是有意的使用。这只是一个错误。所以，我认为抓住这种情况是非常值得的。所以，这在一定程度上取决于具体的情况。

So, I think we spent 40k in a week on that one thing. And so, when it gets to that, again, I think we're pretty liberal as a company and trust people, but I know that's probably not worth it. And so, that's just an example of, again, it wasn't malicious. It wasn't even intended usage really. It was kind of just a mistake. So, I just think catching that is well worth it. So, it depends a bit on the side of the spectrum.

正如你提到的，占工资的 30%。

30% of salary as you mentioned.

那是个粗略估计。是的，

That's a guesstimate. Yeah,

粗略估计。完全没问题。两年后那会是多少？

guesstimate. Totally fine. What is that in two years?

我认为估算编程智能体支出的难点在于，组织正在快速变化。我的意思是，我认为未来组织会小得多，编程智能体支出会与人类工资相同或更高，甚至高一个数量级，而你有一小部分人正在以极快的速度和极高的性能驱动和执行。我认为这是一个诱人的愿景。我不认为包括我们在内的许多公司可能已经达到那个阶段。我认为我们仍然看到，我们需要一群不同背景、不同专业知识的人协作，有大量的沟通、开销和审查，是的，理想情况下是一个人驱动 100 个智能体，驱动构成产品很大一部分的所有事情，但我认为我们仍然看到它仍然是一个非常人性化的组成部分。所以我花了很多时间思考组织设计，是否有可能拥有小团队、一人团队、两人团队来驱动智能体。我认为这非常诱人，在那种情况下，我会说智能体支出可能是成本的三倍，但能够做到这一点的工程师非常有价值，所以也许实际上支出会保持不变，因为能够真正做到这一点并驱动所有那些智能体的人更少，但你付给他们更多，所以支出实际上保持不变。所以我认为组织设计现在非常有趣，因此我认为实际上很难回答那个问题。

I think the hard part about estimating coding agent spend is that orgs are just changing so rapidly. So what I mean by that is I think there's a future where orgs are much smaller and coding agent spend is the same as human salary or more or an order of magnitude more and you have a small number of people that are driving and executing incredibly rapidly and incredibly performantly and I think that's an enticing vision in terms of what you think about. I don't think many companies including us are probably there. I think we still see that we need a bunch of different people with a bunch of different backgrounds and a bunch of different expertises collaborating on things and there's a bunch of communication and overhead and review and like yes it would be ideal to have one person driving a 100 agents and just like driving everything that makes up a large part of the product but I think we just see that it's still a very human component to it. And so I spend a lot of time thinking about org design and is it possible to have small teams, one person teams, two person teams that are driving agents. I think it's very tempting and in that situation I would say agent spend maybe 3x the cost of but then also the engineers who can do that are incredibly valuable and so maybe it actually will just stay the same because the people who can actually do that and who can drive all those agents there's fewer people but you're paying them a lot more and so the spend actually stays the same. So I think org design is very interesting right now and so because of that I think it's actually really hard to answer that question.

对我来说，那个数字如此核心，因为它是 Anthropic 企业价值的根本决定因素。如果那个数字降到 5%，那么 2 万亿美元的 IPO 可能有点定价过高。如果那个数字实际上升到 300%，就像在 Mccor 的案例中那样，那么它就是严重低估。所以这就是为什么我如此关注它。你同意那个数字支撑着 Anthropic 的价值吗？

To me that number is so central because it is the fundamental determinant of Anthropic's enterprise value. If that number goes to 5% then the $2 trillion IPO might be a little bit overpriced. If that number actually goes to 300% like it is in Mccor's case then it's dramatically underpriced. And so that's why I focus so much on it. Do you agree that number underpins Anthropic's value?

我认为那个数字支撑 Anthropic 的价值可能是对的。我认为棘手的是，到目前为止，我们只看到智能体在软件工程领域真正产生影响，而企业中还有许多其他领域我认为尚未受到智能体的影响。所以我认为这是可能的，再次，软件工程我认为很难预测，但我认为这里基本上有两个杠杆。一个是软件工程中会发生什么，是跟今天一样吗？是更多还是更少，但我认为毫无疑问，对于其他领域，它会比今天多得多。所以我认为从这个角度来看，这对 Anthropic、OpenAI 和其他模型提供商非常看好。

I think it's probably right that that number underpins Anthropic's value. I think the tricky part is that the only place we've seen agents really have effect is in software engineering so far though and there are a ton of other places within an enterprise that I would argue have yet to be affected by agents. And so I think it's possible again software engineering I think very difficult to predict but I think there's basically two levers here. One is what happens in software engineering is it the same as today? Is it more than is it less than but I think undoubtedly for the other domains it will be a lot more than it is today. And so I think from that perspective I think that's very bullish on Anthropic and OpenAI and other model providers.

我认为很多人问的另一个问题是关于开源模型的激增以及它如何改变格局。你与一些非常大的企业合作。我的一位好朋友，来自 Saster 的 Jason Lmin 说，我不会模仿他的口音，因为我会搞砸，我以前做过，他说不太好。你发现大型企业害怕使用开源中国模型，还是实际上他们不介意？

I think another question that a lot of people are asking is about the proliferation of open models and how that changes the landscape. You work with some very large enterprises. One of my dear friends Jason Lmin from Saster says that I'm not going to do his accent because I'll butcher it and I've done it before and he got rather he said it wasn't very good. Do you find that large enterprises are fearful of working with open source Chinese models or actually they don't mind?

我们看到一些企业害怕使用开源中国模型。我们看到其他企业不在乎。

We see that some enterprises are fearful of working with open source Chinese models. We see others that don't care.

你认为这种恐惧是合理的吗？

Do you think that fear is justified?

我真的不认为这种恐惧是合理的。我认为有不同的担忧。我认为一个担忧是，它运行的数据会以某种方式回到中国实验室。再次，如果你使用 fireworks 或自己托管，我认为那简直是荒谬的。我认为还有其他恐惧，即他们训练的内容会导致它以不符合企业价值观的方式回应。我认为这就是评估的地方，我认为这很大程度上是没有根据的，但当然也许对于某些领域，那里有一些真实性。我认为这很大程度上是没有根据的，但对于某些领域，由于那种回应，我可以看到那里有一些细微差别。

I don't really think that fear is justified. I think there's different concerns. I think one concern is that the data that it runs on will somehow go back to the Chinese labs. And again, if you're using fireworks or hosting it yourself, I think that's just ridiculous. I think there's other fears that the stuff that they've been trained on will cause it to respond in a way that's not consistent with the values of the enterprise or something like that. I think that's where eval comey and I think that's largely unfounded, but sure maybe for some domains there's some realness there. I think it's probably largely unfounded but for some domains because of that kind of response I could see there being some nuance there.

你认为在未来 12 个月内，我们会拥有可比的美国开源模型吗？

Do you think in the next 12 months we'll have comparable US open source models?

我认为在未来 12 个月内，我们绝对会拥有可比的美国开源模型。我认为你看看 Nvidia 为与生态系统合作所做的一切。我认为他们非常关心这一点。我认为你看看 Reflection 最近发布了一个模型。我认为在某些基准上它仍然稍微落后于一些中国模型，但我认为那个领域有大量的关注，所以我认为绝对会赶上。

I think in the next 12 months we will absolutely have comparable open source US open source models. I think you look at everything that Nvidia is doing to work with the ecosystem. I think they care about it tremendously. I think you look at reflection had a model launch recently. I think it was still a little bit behind some of the Chinese models on some benchmarks but I think there is a tremendous amount of attention in that domain and so I think absolutely those will catch up.

我能问你，当你审视当今开源模型的格局时，鉴于你的职位，你非常清楚地看到了什么世界没有看到的东西？

Can I ask you when you look at the state of landscape today in terms of open models what does the world not see that you see very clearly given your position?

我认为当谈到开源模型时，我认为它们有很多用途，但我确实认为它们用于比人们想象更狭窄的任务。所以我认为我们看到很多人使用更多的开源模型和更多的后训练模型，比我一年或一年半前可能预期的要多，但我仍然认为它们主要用于这些更简单的任务，如路由、护栏、提取或摘要。我不认为它们真正驱动了许多这些主要智能体的核心智能体循环。

I think when it comes to open models I think there is a lot of usage of them but I do think they're for more narrow tasks than people think. So I think we see a lot of people using more open source models and more post-trained models than I would have maybe expected a year or a year and a half ago, but I still think they're primarily for these simpler tasks like routing or guard rails or extraction or summarization. I don't think they're really driving the core agentic loops for a lot of these main agents.

所以我认为开源模型有点分裂，因为有些人站在一端，他们说很好，它们足够好了，让我们把它们用于一切，而我不认为它们已经达到那个水平。然后我认为还有另一些人，他们说你知道前沿模型只会变得越来越好、越来越便宜，是的没错，但如果你真的知道你要做什么，那么你必须有高容量的用例，如果其中有一些组件是足够简单的任务，我认为开始为那些用例进行后训练或类似的后训练模型是非常好的 ROI。所以，我会说，不幸的是，这可能是一个微妙的立场，介于我觉得 Twitter 上的极端观点之间。

And so I think open source models are kind of divisive because you have some people on one end who are like great they're good enough let's use them for everything and I don't think they're quite there and then I think you have other people who are like you know the frontier models are just going to keep on getting better and better and cheaper and cheaper and like yes that's true but if you really know what you're going to do so you have to have a high volume use case and if there are a few components of that that are simple enough tasks I think it's very good ROI to start kind of like post-training or yeah post-training models for those use cases. So, I'd say, you know, unfortunately, it's probably like a nuanced position that's somewhere in the middle of what I feel like the extremes on Twitter are.

你如何看待推理市场的发展？你公开提到了 Fireworks。我们是 Fireworks 的大投资者。我喜欢 Lynn 和团队，但每个人都在尝试做这件事。你我们有 Chase 从 Crusoe 来上节目，他们也将此作为业务的一部分。你知道，我和 Nebius 的人是朋友。他们也将其作为 Rebbit 的一部分。这个业务如何发展，你如何看待做整个技术栈与像 Fireworks 或 Baseten 这样的推理提供商之间的需求？

How do you see the inference market playing out? You mentioned that Fireworks very openly. We're a big investor in Fireworks. I love Lynn and the team, but everyone's trying to do it. You have we had Chase on the show from Crusoe who also have that as part of their business. You know, I'm friends with the Nebius guys. They also do it as part of Rebbit. How does this business play out and how do you think about the need to do all of the stack versus inference providers like a Fireworks or a Baseten?

我认为当涉及到推理方面的事情将如何发展时，有两件事我从 Lang Chain 开始到现在真的低估了。一是对推理的需求，二是对编码智能体的需求。我认为这两个市场比我在两、三、四年前想象的要大得多。所以我认为实际上是 Benchmark 的 Eric Visher 最近在推文或播客上说了这个，我认为他基本上说一切都会成功，这个市场太大了。我认为 Thomas Tungst 也有类似的说法，关于推理市场如此之大，你知道我认为 Eric 说的是更广泛的事情而不仅仅是推理，但我认为应用到推理上，我认为任何推理都会成功。就像，你知道，对它的需求如此之大，我不确定它是否会很快减少。所以，是的，当然，Fireworks 会成功，AWS 会成功，Crusoe 会成功。我只是认为对它的需求如此之大。

I think when it comes to how things will play out with respect to inference, one of the things that I there's two things that I think I really underestimated over the from the start of Lang Chain to now. One is the demand for inference and two is the demand for like coding agents. I think both of those markets are just tremendously bigger than I thought two, three, four years ago. And so I think actually it's Eric Visher at Benchmark who said this on either a tweet or podcast recently where I think he basically said like everything just works like this market is so big. I think Thomas Tungst had something around just like the market for inference is just so big that you know I think Eric said it about more broad stuff than just inference but I think applying it to inference I think any inference will work. That's like, you know, there's just so much demand for it and I don't know if it's going to get less anytime soon. And so, yeah, sure, Fireworks will work, AWS will work, Crusoe will work. I just think there's so much demand for it.

你今天不会投资哪里，比如我可以给你一些例子。

Where would you not be investing today like I so I can give you some examples.

我认为客户支持领域非常无趣。20 家提供商在过去 18 个月内都获得了超过 1 亿美元的资金。你还有 Salesforce 和 Atlassian 等大公司也在那里。你还有像 Intercom 和 Talkdesk 这样稍微老一点的公司在做。好吧，酷。或者像机器人技术，大胆的观点，但有很多钱投入机器人技术。我们离拥有 ChatGPT 时刻还很远。尤其是人形机器人在形态因素上真的很挣扎。我只是认为资金和兴奋程度显著超过了能力，这就是我的答案。

I think the customer support space is like wildly uninteresting. 20 providers all funded with over 100 million in the last 18 months. You got Salesforce and Atlassian and big incumbents there too. You got kind of mildly old ones like Intercom and Talkdesk doing it too. Okay, cool. Or like robotics, like bold take, but there's so much money going into robotics. We're massively far away from having a chat GPT moment. Humanoids in particular really struggle in form factor. I just think the funding and the excitement exceeds the capabilities significantly would be my answer.

我喜欢认为自己有点乐观，所以我认为我确实认为这项技术是变革性的，因此我认为会有大量机会。我想所以我可能会回答那个，然后通过那个我们可以反推出我不会投资什么。我想我认为推理我们已经谈过了。我认为有巨大的有巨大的

I like to think I'm somewhat of an optimist and so I think I do think this technology is transformative and so I think there will be ton of opportunity. I think and so I might answer that and then through that we can back out what I wouldn't invest in. I like I think inference we already talked about. I think there's a tremendous there's tremendous

让我们这样做。你是我们基金的最新合伙人。欢迎。很高兴在你第一天就有你。第一，我们应该有推理。

Let's do this. You're my newest partner in the fund. Welcome. It's so good to have you on your first day. Number one, we should have inference.

我们应该有推理。将会有更多的软件被编写，所以我们也应该投资于更低层次的基础设施原语。Modal，了不起的公司。Temporal，了不起的公司。我认为

We should have inference. There's going to be way more software written, so we should also invest in like lower level kind of like infrastructure primitives. Modal, incredible company. Temporal, incredible company. I think

他们与 Fireworks 竞争。

They compete with Fireworks.

Modal 在某些方面是的。是的。

Modal does on some things. Yeah.

好的。

Okay.

但他们也做更多。他们做沙盒。所以我们有推理层，第一层编码层

But they also do more. They do sandboxes. So we've got inference layer one the coding layer

编码呃或者我会说推理然后像基础设施，所以像低级基础设施 Temporal Modal 嗯 Superbase Convex 嗯垂直智能体我认为垂直智能体将是一个好的层，我认为嗯

Coding uh or I would say inference and then like infrastructure so like low-level infrastructure temporal modal um uh superbase convex um vertical agents I think vertical agents are going to be a good layer I think um

任何特定的垂直领域

Any specific verticals

我不知道是否有任何垂直领域我会如此超配。我认为重要的部分是拥有特定领域的观点并真正深入那里。我认为 Patrick Collison 在最近的一个播客上说了这个。我认为与 Replit 的 Omrad 一起，当他谈论下一个他会建立什么初创公司时，可能不会是一个性感的领域。它会是一些技术或至少 AI 渗透率低但那里积累了很多领域专业知识的领域，然后进入那里并真正改变它。我认为关键部分基本上就是拥有一些领域专业知识，这些知识可以捕获在数据或逻辑中，放入工具链，最终可能放入模型权重，但我不会从那里开始，拥有一些可以捕获的领域专业知识，然后将其投入使用。

I don't know if there's any kind of like verticals that I would overweight so much. I think the important parts are having like a domain specific kind of point of view and really diving deep there. I think Patrick Collison said this on a recent podcast. I think with Omrad from Replit when he was talking about like what the next what startup he would build it's probably wouldn't be like a sexy area. it would be some area where like technology maybe or at least AI penetration is low but there's a lot of domain expertise built up there and kind of going in there and really transforming that. I think the key part is basically just having some domain expertise that can either be captured in data or logic that goes in the harness and eventually maybe the model weights but but I wouldn't start there having some domain expertise that can be captured there and then putting it to work.

顺便说一句，我认为在所有这些都是我可能会寻找的大概三件事，我认为我们的 prerex 如果你看看所有如果你看看所有这些成功的团队，第一，我认为他们拥有的人才非常强大，就像 Modal 有一个令人难以置信的强大人才团队。Lynn 和 Fireworks 对吧，他们编写了 PyTorch，那有多酷？第二

I think by the way there are maybe like three things that I would look for in all of these and I think our prerex if you look at all of the if you look at all of these teams that are succeeding one I think like the talent that they have is tremendously strong like like Modal has an incredibly strong kind of like talent team. Lynn and Fireworks right they wrote PyTorch like how cool is that right? Two

当我们见到他们时，你知道当你见到创始人时，最后一次你做简短介绍时，就像我构建了 PyTorch

When we met them you know when you're kind of meeting founders the last time you do the brief intro it's like I built PyTorch

那有效。是的。是的。那个框里打了好勾。

That works. Yeah. Yeah. Good tick in that box.

所以我认为要构建特别是这种真正深入的基础设施，你必须在技术上非常强大，而能做到这一点的人并不多。第二，你必须非常非常快，就像能够快速迭代，我认为这非常重要。市场变化如此之快。我认为速度和执行速度，当我想起我们在 LangChain 的优势时，我认为我们整合得非常非常快，我认为这是我们对其他公司的优势之一，最后一点是品牌，我认为像很多在这个 AI 领域 AI 时代成功的公司基本上抓住了闪电，然后乘着那股浪潮并抓住了它。你看看早期的 LangChain，我们非常病毒式传播，并在此基础上执行。

So I think to build like especially this really deep infrastructure you have to be like super technically strong and there's just not that many people that can do that. Two you have to go like really really fast like this like being able to iterate quickly I think is incredibly important. The market just changes so rapidly. I think speed and execution speed when I think about our advantages at LangChain I think we integrate really really quickly and I think that's one of the advantages we have over other companies and then the last thing would be brand I think like a lot of companies that you know made it in this AI area AI era basically caught lightning in a bottle and then like rode that wave and captured that. You look at like LangChain early on like we went very viral and have executed against that.

Cursor 是个很好的例子——他们迅速走红，并借此执行。甚至 Claude Code、GPT——感觉这个领域发展得太快了，因为竞争者众多，也许正因为如此，你知道在 LangChain 出现后的两个月里，涌现了上百个不同的框架。但我认为我们拥有积累起来的品牌优势，我认为那是一种持久的优势，我们乘势而上。Instinct 是另一个例子——可能还有很多个人助手。但就融资轮次而言，估值可能高于理性水平。但对于一些正在完成的融资轮次，如果有一家突破性公司拥有非常强大的技术团队，并证明他们能快速执行，我认为你就不看价格，直接投资。

Cursor is a great example—they went very viral and executed against that Jev. Even Claude Code, GPT—it seems like the space is moving so fast, and because there are so many competitors, maybe even because of that, you know there were like a hundred different frameworks that came out in the two months after LangChain came out. But I think we had a brand advantage that was built up, and I think that was a durable advantage that we rode. And so Instinct is another example—probably a bunch of personal assistants. But in terms of funding rounds, it would probably be at higher valuations than is rational. But for some of these funding rounds that are getting done where there's a breakout company with a really strong technical roster that has proven they can execute quickly, I think you just don't look at the price and you just invest.

成为第一是否比以往任何时候都更重要？

Is being first more important than ever before?

我认为是的。绝对如此。是的，你看——我的意思是追赶是可能的。我认为 OpenAI 在追赶 Codex 方面做得非常出色。但成为第一——Anthropic 和 Claude Code——哇，那是多大的优势啊。

I think so. Absolutely. Yeah, you look at—I mean it's possible to catch up. I think OpenAI does an incredible job of catching up with Codex. But being first—Anthropic and Claude Code—wow, what an advantage that was.

但正如你所说，LangChain 出现后，几个月后越来越多的出现。但 Instinct 也一样——但实际上，作为第一个光环带来的持久品牌垄断是非常真实的。

But to your point, LangChain comes out and then months after more and more come out. But Instinct same—but actually the durable brand monopoly that comes from being the halo first is very real.

我认为那非常真实。我也认为在那之后你必须执行得非常好。对我刚才所说的话的一个反例：仅仅因为你是第一个，并不意味着你保证成功。AutoGPT——我不知道你是否记得 AutoGPT 项目——但它在 LangChain 之后几个月出现，是有史以来增长最快的 GitHub 项目。从那以后我就没怎么听到他们的消息了。这不是贬低——快速执行非常困难。但我认为仅仅因为你在某个领域是第一个，并不意味着你能生存下来。我从不喜欢贬低别人，所以我觉得在某些方面赞扬他们反而更有帮助。

I think that is super real. I also think that you have to execute incredibly well after that. A counterexample to what I just said: just because you're first doesn't mean that you're guaranteed success. AutoGPT—I don't know if you remember the AutoGPT project—but it came out a few months after LangChain and was the fastest growing GitHub project of all time. And I haven't heard much from them since. And that's not a knock—it's very hard to execute with speed. But I think just because you're first in an area doesn't mean that you'll survive. I never like to knock people, so I find it helpful to praise them instead in some respects.

如果你要写一份加入公司的备忘录，说明为什么我们应该投资 90 亿美元的 Jev 轮，你对 Jev 的看涨理由是什么？

If you were to write a memo joining the firm for why we should invest in a $9 billion Jev round, what would your bull case be for Jev?

Jev 的看涨理由是，有一种新型推理和新型模型，而模型类别实际上以他们命名。所以他们将乘上这股浪潮。而且他们是一支非常有才华的团队，能够执行。我不太了解这个团队。根据我所看到的一切，对于那个估值的轮次，这必须是一个假设。情况似乎如此，但这就是看涨理由。他们基本上发明了一个新的模型类别，与之关联，并拥有一支有才华的团队能够执行。

The bull case for Jev is that there's a new type of inference and a new type of model, and the model category is literally named after them. So they're going to ride that wave. And they're a very talented team that can execute on that. I don't know the team super well. I think that would have to be the assumption for a round at that value from everything that I've seen. That seems to be the case, but that would be the bull case. They basically invented a new model category, are associated with it, and have a talented team that can execute on that.

从 Benchmark 拿钱会改变一家公司的轨迹吗？

Does taking money from Benchmark change the trajectory of a company?

100%。我认为 Benchmark——当 Miles 和整个 Benchmark 合伙团队投资我们时，只有我和 Ankush。我们只有两个人。我们真的把他们当作第三位联合创始人。我认为——你知道，当我想到从两个人到现在大约 400 人的公司建设时，他们肯定是那段旅程中最始终如一的合作伙伴。你知道，我不是风投。我从未做过风投。可能还有其他市场动态，伟大的 Benchmark 参与其中发出了信号。我们非常幸运在 Benchmark 轮之后大约 2、3 个月有 Sequoia 加入我们。他们也是一个了不起的合作伙伴。

100%. And I think Benchmark—when Miles and really the whole Benchmark partnership invested in us, it was me and Ankush. We were two people. We really thought of them as a third co-founder. I think there are—you know, when I think about company building from two people to where we are now today, which is like 400 people, they've been our most consistent partner in that journey for sure. You know, I am not a VC. I've never done VC. There's probably also other market dynamics where great benchmarks involved in that signals thing. And we're very lucky to have Sequoia join us after like 2 months, 3 months after the Benchmark round. And they've been a tremendous partner as well.

Sequoia 那边是谁？

Who was it from Sequoia?

Sequoia 的 Sonia，我认为她是这个领域人脉最广的人之一。

Sonia from Sequoia, who is one of the most plugged-in people in the space, I think.

他们的风格有何不同？

How do their styles differ?

所以他们都非常聪明、敏锐，并且带来了真实经验。我认为 Benchmark 可能更侧重于——公司的核心是什么——他们将成为这方面的共鸣板，并在这方面提供帮助。所以我的意思是，很多公司建设、定位的事情，我会去找他们，他们会花时间在那上面。Sequoia 的风格我认为是他们帮助一切——他们就是一台机器。所以他们帮助招聘个人贡献者、高管，他们帮助营销和发布，他们真的会运营化。我想是的。是的。我的意思是我们从 Sequoia 的推荐中得到了很多候选人。

So they're both very intelligent and sharp and have real experience that they bring. I think Benchmark is maybe a little bit more like—what's the core nucleus of the company—and we're going to be a sounding board for that and we're going to help with that. So what I mean by that is, a lot of the company building things, positioning things, I would go to them for and they would spend time there. Sequoia's style I think is they help with everything—they are just a machine. So they help with recruiting of ICs, of executives, they help with marketing and launches, and they'll really operationalize. I think so. Yeah. I mean we've gotten a lot of our candidates from Sequoia references.

是的。他们有一个人才网络，我们从那里得到了很多非常强的候选人。

Yeah. They have a talent network and we've gotten a bunch of really strong candidates from there.

你会对创始人说值得为 Sequoia 接受折扣吗？

Would you say to founders it's worth taking a discount for Sequoia?

我认为绝对值得为 Sequoia 和 Benchmark 以及可能其他顶级或一线风投公司接受折扣，但那些是我们合作的。

I think absolutely it's worth taking a discount for Sequoia and Benchmark and probably other top-tier or tier-one VC firms, but those are the ones that we work with.

关于融资，你现在知道什么，希望当初只有你们两个人时能告诉自己？

What do you know now about fundraising that you wish you could tell yourself when it was just the two of you?

我认为融资几乎就像一场游戏。我一开始肯定不知道。我是第一次创业。我也认为我非常专注——我所有时间基本上都花在产品上——所以我非常专注于此。我确实认为融资有这种游戏，你想不融资就能融资。你想——一旦你拿到一份条款清单——你想拿到一份条款清单，然后告诉别人你有条款清单，然后

I think there is like this game to fundraising almost. And I definitely didn't know that going in. I'm a first-time founder. I also think I focus a lot—where I spend all my time is on product basically—and so I'm very focused on that. I do think there is this game to fundraising where you want to fundraise without fundraising. You want to—once you get one term sheet—you want to get one term sheet and then tell others you have a term sheet and

在 Twitter 上发布收入图表，然后就会带来所有这些 inbound 和

post a revenue graph on Twitter and then it leads to all these inbound and

是的。

yeah.

你喜欢这个吗？我讨厌它。也许我现在太老了，不适合这个游戏，但这只是装腔作势。就像 Harrison，你真的喜欢我，想和我一起创业吗？我会给你同样的价格和同样的钱，不管怎样。就像我们就这样做吧。停止装腔作势。

Do you like this? I hate it. Maybe I'm too old for this game now, but like it's just posturing. It's like Harrison, do you really like me and want to build a business with me? I'll give you the same price and the same money than whatever. Like let's just do this. Stop the posturing.

我认为我一点也不喜欢融资。我认为——我的意思是就像你说的——我想和我喜欢的人一起构建伟大的东西。我认为是的，特别是对于我们拥有的雄心规模，我们需要融资来实现。我认为那是 Benchmark 在我们向他们融资时采取的方法——就像嘿，让我们弄清楚我们是否喜欢对方并想一起工作。顺便说一句，那是他们非常好的谈判。所以我认为他们也是非常好的谈判者。

I think I don't like fundraising at all. I think—I mean kind of what you were saying—I want to build something great with people I enjoy working with. I think yeah, especially for the scale of ambitions that we have, we need to fundraise to get that done. I think that was the approach that Benchmark kind of took when we were fundraising from them as well—like hey let's figure out if we like each other and want to work together. By the way, that was very good negotiating by them. And so I think they're very good negotiators as well.

但回头看我们最近完成的几轮融资，比如我们 B 轮从 IVP 的 Tom 那里拿到的投资——我从红杉那轮前后就开始和 Tom 聊，聊了两年半多。所以那是一段我们随着时间慢慢建立起来的关系，我了解了他，他也了解了我。所以到了真正融资的时候，我觉得里面的套路少了很多，我真的很感激这一点。

But when I look at our most recent rounds that we've done, like we raised from Tom at IVP, our Series B — I'd been chatting with Tom from right around the Sequoia round for over two and a half years. So that was a relationship that we just built up over time, and I got to know him and he got to know me. And so when it came time to fundraise, I feel like there were fewer games involved there, and I really appreciated that.

我想来个快问快答。我说一句简短的话，你直接给我第一反应。可以吗？

I would love to do a quickfire round. So I say a short statement and you give me your immediate thoughts. Does that sound okay?

来吧。

Let's do it.

好。你最喜欢的一次 VC 会议是哪次？

Okay. What was your single favorite VC meeting?

和 Benchmark 的第一次路演。

First pitch with Benchmark.

为什么？

Why?

满屋子都是传奇人物。

So many legends in the room.

跟谁相处最酷？

Who is the coolest to hang with?

Bill Gurley。

Bill Gurley.

你当时有被震慑到吗？

Were you intimidated?

我觉得我当时太天真了，根本不知道怕。

I think I was too naive to be intimidated.

他本人也是个大块头，那种居高临下压过来的气势很吓人。关于智能体，你相信但生态里大多数人不会同意的最重要的一点是什么？

He's also just like a big man, like it's an intimidating figure looming over you. What is the single most important thing you believe about agents that most of the ecosystem would disagree with?

做智能体依然非常难，真的非常非常难。也许大家会同意这一点，但我觉得它的难度被严重低估了。

It's still really hard to build them, like really, really hard. Maybe people would agree with that, but I think it's just underrated how hard it is.

过去 12 个月里，你改变看法最大的是什么？

What have you changed your mind on most in the last 12 months?

推理市场非常巨大。我记得一年半前和董事会讨论要不要做一个网关，当时我们看不到相比 OpenRouter 这类产品能增加什么价值，我觉得那百分之百是个错误的决定。

The market for inference is massive. I remember talking with the board about whether we should do a gateway a year and a half ago, and we didn't see what value we could add over something like OpenRouter, and I think that was 100% the wrong decision.

你和团队做过的最难的技术工作是什么？

What's been the hardest technical thing for you and the team to build?

我们自己造了一个用于存储智能体轨迹的数据库，Smith DB。

We built our own database for storing agent traces, Smith DB.

你怎么看待既和 ClickHouse 竞争、又是它的客户这件事？是有点奇怪还是很正常？

How do you think about competing with ClickHouse and being a customer? Kind of weird or normal?

我觉得我们有 500 家这种亦敌亦友的公司，所以我个人已经非常习惯了。

I think there are 500 different companies that we are frenemies with, and so I think I've personally gotten very used to it.

你可以投一个 IPO：OpenAI 估值 1.5 万亿，或者 Anthropic 估值 2 万亿。

You can invest in one IPO: OpenAI at 1.5 trillion or Anthropic at 2 trillion.

OpenAI。

OpenAI.

为什么？

Why?

我觉得他们有更多元、不同的路径通向真正惊人的退出估值。

I think they have more diverse different paths to really tremendous exit valuations.

完全理解。有什么是你现在知道、但希望 2022 年刚创业时就知道的？

Totally get that. What do you know now that you wish you'd known when you started in 2022?

我希望当时就知道今天的 harness 长什么样，这样我们就能更早把它们做出来。

I wish I knew what harnesses today looked like so we could have built them earlier.

最后一个问题。展望五年后，你最期待的是什么？

Final one for you. What are you most excited for when you look forward 5 years?

我非常期待我们能用 AI 做出哪些有创意的东西。很多 AI、我们做的很多工作，都是帮企业提高效率、交付成果。我觉得像 Suno 这样的创意产品非常有意思，我希望看到更多这类东西。

I'm really excited for what creative things we can do with AI. A lot of AI, a lot of what we work on is helping enterprises be more efficient and delivering outcomes. I think creative things like Suno are tremendously interesting, and I wish we'd see more of those.

完全同意。我记得请过 Mikey 上节目，那期其实挺有意思的。这次聊得特别开心。Harrison，非常感谢你包容我的随性和不规律的日程，你是一位很棒的嘉宾。

Totally agree. I remember having Mikey on the show. That was a fun one, actually. And this has been so much fun to do. Harrison, thank you so much for putting up with my waywardness and divergent schedules, but you've been an amazing guest.

不，非常感谢你邀请我。

No, thank you so much for having me.