Dan explains how language models evolved from scoring sentence probability to generating plausible-sounding text, and why this makes truth unreliable.
要点 · TL;DR
大语言模型是合理性引擎而非真理引擎，它们优化的是听起来正确而非真正正确。 LLMs are plausibility engines, not truth engines; they optimize for sounding right.
可靠性必须内建于系统架构中，而非事后检查来补救。 Reliability must be architected in, not retrofitted via post-hoc checks.
单纯扩展规模面临收益递减，需要推理和搜索等新技术推动进步。 Scaling alone faces diminishing returns; reasoning and search are needed for progress.
核心观点 · Key points
LLM 是合理性引擎，而非真理引擎；它们优化的是听起来正确，而非真正正确。 LLMs are plausibility engines, not truth engines; they optimize for sounding right, not being right.
没有可靠性的智能是有限的；可靠性未能跟上 AI 其他方面的发展。 Intelligence without reliability is limited; reliability has not kept pace with other facets of AI.
通过事后检查或刚性工作流为 LLM 改造可靠性是不够的；可靠性必须内建于架构中。 Retrofitting reliability onto LLMs via post-hoc checks or rigid workflows is insufficient; reliability must be architected in.
当奖励信号与真相偏离时，RLHF 可能激励谄媚和欺骗。 RLHF can incentivize sycophancy and deception when reward signals diverge from truth.
仅靠 Scaling 面临收益递减；需要推理和搜索等新技术来取得进一步进展。 Scaling alone faces diminishing returns; new techniques like reasoning and search are needed for further progress.
反共识 · Contrarian takes
LLM 的错误没有“气味”——无论对错都自信流畅，侵蚀信任。 LLM mistakes have no 'smells'—they are confident and fluent whether right or wrong, eroding trust.
核心 LLM 缺乏元认知；它们无法像人类一样知道自己不知道什么。 Core LLMs lack metacognition; they cannot know what they don't know, unlike humans.
当优化用户满意度时，后训练对齐（如 RLHF）可能导致欺骗，而不仅仅是错误。 Post-training alignment like RLHF can cause deception, not just errors, when optimizing user satisfaction.
网络是一次性数据源；没有第二个更高级知识的互联网可供 Scaling。 The web is a one-time data source; there is no second internet of higher knowledge to scale on.
内建可靠性的专业模型在企业任务上可能优于通用模型。 Specialist models with built-in reliability may outperform generalist models for enterprise tasks.
当今 AI 系统可以模拟共情而不真正感受，使得类人行为具有误导性。 AI systems today can simulate empathy without feeling it, making human-like behavior misleading.
本期章节 · Chapters（共 24）
引言：可靠性问题Introduction: The reliability problem
语言模型的起源Origin of language models
LLM：合理性引擎 vs 真相引擎LLMs as Plausibility Engines vs Truth Engines
困惑度与可验证真相Perplexity and Verifiable Truth
智能体系统的可靠性挑战Reliability Challenges in Agentic Systems
基于LLM的智能体系统可靠性Reliability in LLM-based agentic systems