Engram co-founders discuss how models can learn continuously from evolving contexts, moving beyond pre-training and post-training to deeply integrate new knowledge into model weights.
要点 · TL;DR
模型有用性的瓶颈在于学习新上下文，而非原始智能。 Model usefulness bottleneck is learning new context, not raw intelligence.
记忆应内化到权重中，而非仅靠外部检索。 Memory should be internalized into weights, not just external retrieval.
未来 AI 是众多个性化模型，而非一个巨型前沿模型。 Future AI is many personalized models, not one giant frontier model.
核心观点 · Key points
模型实用性的瓶颈不是原始智能，而是学习新的、不断变化的上下文。 The bottleneck for model usefulness is not raw intelligence but learning new and evolving context.
记忆应内化到模型权重中，而不仅仅通过上下文或检索外化。 Memory should be internalized into model weights, not just externalized via context or retrieval.
持续学习需要新的研究突破、小模型基础设施以及研究与产品的集成循环。 Continual learning requires new research breakthroughs, infrastructure for small models, and integrated research-product loops.
事实和技能不可分离；内化事实能实现更深入的组合和推理。 Facts and skills are not separable; internalizing facts enables deeper composition and reasoning.
为每个团队或用户训练模型，相比 RAG 可将推理成本降低几个数量级。 Training per-team or per-user models can reduce inference costs by orders of magnitude compared to RAG.
反共识 · Contrarian takes
未来不是一个巨大的前沿模型，而是为每个个人或团队定制的众多模型。 The future is not one giant frontier model but many personalized models for every individual or team.
将知识内化到权重中能实现检索系统无法实现的联想回忆。 Internalizing knowledge into weights enables associative recall that retrieval systems cannot achieve.
KV 缓存效率低下；一篇小文章可能消耗 80GB GPU 内存，而模型权重压缩了整个互联网。 KV cache is inefficient; a small article can consume 80GB of GPU memory, while model weights compress the entire internet.
语言模型主导视觉并非因为固有优势，而是由于电子处理的对等性。 Language models dominate vision not because of inherent superiority but due to electronic processing parity.
梦境启发 AI 训练：模型应从交互中退下，离线实验其能力范围。 Dreams inspire AI training: models should retreat from interaction to experiment with affordances offline.
本期章节 · Chapters（共 11）
0. Engram与持续学习简介Introduction to Engram and continual learning
1. 定制Notion代理的前提Premise of custom Notion agent
2. 上下文与模型的融合Convergence of context and model
3. 研究与产品整合Integrating Research and Product
4. 苦涩教训与计算扩展Bitter Lesson and Compute Scaling
5. KV缓存与推理效率KV Cache and Inference Efficiency
6. 记忆与持续学习的未来Future of Memory and Continual Learning