LangChain 的 Harrison 探讨了掌控智能体框架、模型和上下文的重要性,并解释了如何通过中间件定制核心智能体循环。
Harrison from LangChain discusses the importance of owning your agent's harness, model, and context, and explains how to customize the core agent loop using middleware.
要点 · TL;DR
拥有你的智能意味着拥有控制装置、模型和上下文,而不仅仅是模型。 Owning your intelligence means owning the harness, model, and context, not just the model.
评估和可观测性是调试代理和构建数据飞轮以持续改进的关键。 Evals and observability are key to debugging agents and building a data flywheel for continuous improvement.
从通用控制装置开始以快速见效,然后随着用例的明确而进行定制。 Start with a general harness for speed, then customize as your use case narrows.
核心观点 · Key points
智能体由 harness、模型和上下文三部分组成;拥有全部三部分对于拥有你的智能至关重要。 An agent consists of a harness, a model, and context; owning all three is key to owning your intelligence.
harness 的主要职责是在正确的时间将上下文提供给模型,编排模型调用和工具调用的循环。 The harness's main job is to bring context to the model at the right time, orchestrating the loop of model calls and tool invocations.
从通用 harness 开始以快速获得价值,然后随着用例的聚焦逐步添加定制化。 Start with a general harness for quick time-to-value, then add customizations as you narrow down your use case.
评估和可观测性对于调试智能体以及驱动数据飞轮以累积智能至关重要。 Evals and observability are essential for debugging agents and powering a data flywheel to compound intelligence.
Harbor 正成为定义智能体基准的行业标准,任务在沙箱中运行并由测试评分。 Harbor is becoming the industry standard for defining agent benchmarks, with tasks run in sandboxes and scored by tests.
环境反馈和合成反馈对于整理轨迹数据和持续改进智能体至关重要。 Feedback, both environmental and synthetic, is crucial for curating trace data and improving agents continuously.
反共识 · Contrarian takes
当你的任务在分布内时,现成的 harness 效果最好;任务越偏离分布,就越需要定制 harness。 Off-the-shelf harnesses work best when you are in distribution; the more out-of-distribution your task, the more you need a custom harness.
即使对于分布外任务,也要将小的分布内部分(如文件编辑)保持在模型层附近以获得最佳性能。 Even for out-of-distribution tasks, keep small in-distribution parts (like file editing) close to the model layer for best performance.
智能体失败更多是由于上下文不佳而非模型能力不足,因此对上下文窗口的可观测性至关重要。 Agent failures are more often due to poor context than model inadequacy, making observability into the context window critical.
UX 设计可以在没有明确点赞/点踩的情况下从用户那里获得反馈,这通常更有效。 UX design can elicit feedback from users without explicit thumbs up/down, which is often more effective.
Harness 在编码领域可能趋同,但在生物等专业领域会分化,导致针对特定模型的优化。 Harnesses may converge for coding but diverge for specialized domains like bio, leading to model-specific optimizations.
利用基准,你可以运行不同的 harness 和模型,然后将最佳实践(如“codex 化”)融入自己的系统中。 Using a benchmark, you can run different harnesses and models, then incorporate the best practices (e.g., 'codexification') into your own.
本期章节 · Chapters(共 11)
引言Introduction
智能体构成What makes up an agent
核心智能体循环The core agent loop
定制化工具链Harness customization
评估与可观测性Evals and observability
任务定义与运行Defining tasks and running agents
智能体可观测性Observability for agents
数据飞轮与反馈Data flywheel and feedback
轨迹数据整理与更新Curating trace data and updating the agent