Dan Klein 探讨了基于下一个词预测构建的 AI 系统是可信度引擎而非真理引擎，以及为什么幻觉和收益递减是行业面临的最大挑战。
Dan Klein discusses how AI systems built on next-token prediction are plausibility engines, not truth engines, and why hallucinations and diminishing returns are the industry's biggest challenges.
要点 · TL;DR
大语言模型是合理性引擎，而非真理引擎。 LLMs are plausibility engines, not truth engines.
可靠性是当前 AI 系统的关键弱点。 Reliability is the key weakness of current AI systems.
由于数据和算力限制，规模法则正遭遇收益递减。 Scaling laws are hitting diminishing returns due to data and compute limits.
核心观点 · Key points
大语言模型是合理性引擎，而非真实性引擎；它们优化的是流畅输出，而非事实准确性。 LLMs are plausibility engines, not truth engines; they optimize for fluent output, not factual accuracy.
可靠性是当前AI系统的关键弱点，落后于广度、上下文性等其他智能维度。 Reliability is the key weakness of current AI systems, lagging behind other facets of intelligence like breadth and contextuality.
强化学习可能增加幻觉，因为它优化用户满意度而非真实性。 Reinforcement learning can increase hallucinations by optimizing for user satisfaction over truthfulness.
领域应构建以真实性为设计原则的系统，而非在现有模型上事后补救可靠性。 The field should build systems with truth as a design principle, not retrofit reliability onto existing models.
模块化与端到端优化存在张力；如何结合两者是构建可靠AI的核心挑战。 Modularity and end-to-end optimization are in tension; combining them is a central challenge for reliable AI.
反共识 · Contrarian takes
大语言模型的每个输出都是幻觉；有些正确，有些错误——系统缺乏元认知。 Every LLM output is a hallucination; some are right, some are wrong—the system lacks metacognition.
用额外系统事后补救可靠性可能加剧错误，因为错误往往相互关联。 Retrofitting reliability with additional systems can compound errors, as errors often correlate.
AI行业建立在果冻之上——产生合理但不可验证输出的概率引擎。 The AI industry is built on jello—probabilistic engines that produce plausible but unverifiable output.
缩放定律正遭遇收益递减；我们今天已看到数据墙和算力限制。 Scaling laws hit diminishing returns; we are seeing data walls and compute limits today.
尽管端到端学习当前占主导，语言结构可能作为提升可靠性的表示而回归。 Linguistic structure may return as a useful representation for reliability, despite current dominance of end-to-end learning.
本期章节 · Chapters（共 18）
从无效到全效的引言Introduction: From Nothing Works to Everything Works
幻觉与强化学习Hallucinations and Reinforcement Learning
定义幻觉Defining Hallucination
当前系统的元认知Metacognition in Current Systems
可靠性vs.水平性Reliability vs. Horizontality
可靠性新架构A New Architecture for Reliability
提示与猎物：控制幻觉Prompt and Prey: The Illusion of Control
Token与可验证RL数据Tokens vs. Verifiable RL Data
代码异味与AI输出同质化Code Smells and the Homogenization of AI Output
从语言模型到可靠性挑战From Language Models to Reliability Challenges
数字素养与信任线索缺失Digital Literacy and the Loss of Trust Cues
文化未跟上AI速度Culture hasn't kept up with AI speed
两种决策方式Two ways of making decisions
人类语言学启发AIHuman linguistics informing AI
功能负荷假说规模化Functional Load Hypothesis at Scale
错误减少与保证的张力Tension between error reduction and guarantee
语言与LLM的句法分析Syntactic parsing in language and LLMs
Transformer与语言学子结构Substructure in Transformers and Linguistics