Flash Attention 主要作者、Together 首席科学家 Tri Dao 探讨 AI 硬件的未来、NVIDIA 的竞争对手以及模型架构的演变。
Tri Dao, lead author of Flash Attention and chief scientist at Together, discusses the future of AI hardware, competitors to NVIDIA, and the evolution of model architectures.
要点 · TL;DR
自 ChatGPT 以来，推理成本通过量化和协同设计下降了约 100 倍。 Inference costs dropped ~100x since ChatGPT via quantization and co-design.
代理型工作负载将成为 AI 的下一个杀手级应用。 Agentic workloads will be the next killer app for AI.
开源模型将在一年内缩小与闭源模型的差距。 Open-source models will close the gap with closed-source in a year.
核心观点 · Key points
自 ChatGPT 以来，推理成本下降了约 100 倍，这得益于量化、协同设计和更好的内核。 Inference costs have dropped ~100x since ChatGPT, driven by quantization, co-design, and better kernels.
未来的推理提升将来自硬件专业化、模型架构协同设计和内核优化。 Future inference gains will come from hardware specialization, model architecture co-design, and kernel optimization.
智能体式工作负载（工具使用、多步推理）将成为 AI 的下一个杀手级应用。 Agentic workloads (tool use, multi-step reasoning) will be the next killer app for AI.
由于强化学习扩展和工具链的进步，开源模型将在一年内缩小与闭源模型的差距。 Open-source models will close the gap with closed-source in a year, thanks to RL scaling and tooling.
数据质量和合成数据生成被低估，但对模型改进至关重要。 Data quality and synthetic data generation are underhyped but crucial for model improvement.
反共识 · Contrarian takes
硬件可移植性是个神话；即使是 NVIDIA 芯片，每一代也需要重写软件。 Hardware portability is a myth; even NVIDIA chips require rewriting software every generation.
当前的 Transformer 架构可能足以实现 AGI，但更好的架构可以降低成本 10 倍。 Current transformer architecture may be sufficient for AGI, but better architecture could cut cost 10x.
AI 编码工具使嘉宾的效率提高了 1.5 倍，远高于预期。 AI coding tools made the guest 1.5x more productive, much higher than expected.
推理工作负载将分化为三种模式：聊天机器人、低延迟智能体式和高吞吐量批处理。 Inference workload will diversify into three patterns: chatbot, low-latency agentic, and high-throughput batch.
模型在合成数据生成中扮演人类角色时出奇地好，例如扮演愤怒的纽约人。 Models are surprisingly good at acting as humans for synthetic data generation, e.g., angry New Yorkers.
本期章节 · Chapters（共 20）
引言与嘉宾背景Introduction and Guest Background
架构与工作负载多样化Architecture and workload diversification
创业押注与前瞻优化Startup bets and forward-looking optimizations
硬件生态与抽象层Hardware ecosystem and abstraction layers
AI 与硬件可移植性抽象Abstractions for AI and Hardware Portability
利用 AI 生成 GPU 内核Using AI for GPU kernel generation
推理成本降低与 Flash AttentionInference cost reduction and Flash Attention
推理优化进展Inference Optimization Progress
Mamba 与 KV 缓存压缩Mamba and KV cache compression
多样化用例与推理优化Diversifying use cases and inference optimization
三个推理用例Three Inference Use Cases
无尽优化？Endless Optimizations?
未来 AI 基础设施问题Question about future AI infrastructure
目标：实现专家级 AIGoal: achieving expert-level AI
架构创新的作用Role of architectural innovation
有前景的架构：MoE 与状态空间Promising architectures: MoE and state space
当前研究兴趣与未来方向Current research interests and future directions