Modal's CTO shares how they shifted focus from developer to agent experience, built a serverless runtime for bursty compute, and why decorators beat YAML for infrastructure.
要点 · TL;DR
Modal 将重心从开发者体验转向智能体体验，发现两者 90% 相似。 Modal shifted focus from developer to agent experience, finding 90% similarity.
自定义模型的弹性推理是 Modal 最大的用例，自动扩缩容是关键。 Elastic inference for custom models is Modal's biggest use case, with auto-scaling key.
基于块的推测解码可实现 2-4 倍加速，且不损失质量。 Speculative decoding with block-based draft models yields 2-4x speedup without quality loss.
核心观点 · Key points
Modal 将其 SDK 团队的重点从开发者体验转向了智能体体验，因为同样的优势也适用于智能体。 Modal shifted its SDK team focus from developer experience to agent experience, as the same benefits apply.
为定制模型提供的弹性推理是 Modal 最大的用例，其流量模式不可预测。 Elastic inference for custom models is Modal's biggest use case, with unpredictable traffic patterns.
基于块的推测解码草案模型可以在不损失质量的情况下实现 2-4 倍的加速。 Speculative decoding with block-based draft models can give 2-4x speedup without quality loss.
Modal 专注于模型生命周期和智能体生命周期，从数据准备到推理。 Modal focuses on the model lifecycle and agent lifecycle, from data prep to inference.
自动扩缩容和 GPU 快照是 Modal 推理平台的关键差异化优势。 Auto-scaling and GPU snapshotting are key differentiators for Modal's inference platform.
反共识 · Contrarian takes
智能体体验与开发者体验并无本质区别；它们有 90% 的相似性。 Agent experience is not fundamentally different from developer experience; they share 90% similarity.
提供模型 API 服务于爱好者市场，其粘性低于提供代码级灵活性。 Providing model APIs serves a hobbyist market and is less sticky than offering code-level flexibility.
Modal 不拥有数据中心；在软件上保持轻资产可以更快地迭代。 Modal does not own data centers; being capital-light on software allows faster iteration.
智能体权限需要硬性护栏；由 LLM 调解的权限存在风险。 Hard guardrails are needed for agent permissions; LLM-mediated permissions are risky.
视频生成的下一个大事件是智能体编排多个模型以生成长篇内容。 The next big thing in video generation is an agent orchestrating multiple models for long-form content.
本期章节 · Chapters（共 18）
代理体验与开发者体验Agent Experience vs Developer Experience
可观测性 vs 代码审查Observability vs. Code Review
通用型 vs 专用型General-Purpose vs. Specialized
客户协作Customer Collaboration
Modal 个人体验Personal Experience with Modal
沙箱化历程Sandboxing Journey
弹性推理与自动缩放Elastic inference and autoscaling
自动端点：让前沿模型可访问Auto Endpoints: Making Frontier Models Accessible
构建代理的完美原语Building the perfect primitive for agents
网络功能：沙箱、边车与覆盖网络Networking features: sandboxes, sidecars, and overlay networks
分布式训练中的 RDMA 与内存移动RDMA and memory movement in distributed training
多节点训练与网络Multi-node training and networking
计算容量管理与批处理层Compute capacity management and batch tier
Modal 下一阶段与产品原语Next phase of Modal and product primitives
代理权限与 LM 中介控制Agent permissions and LM-mediated control
生态竞争与 Modal 角色Ecosystem competition and Modal's role