智能的物理学：从沙堆到潜在空间

Physics of Intelligence: From Sand to Latent Space

马蒂厄·维亚尔 Matthieu Wyart · ML Street Talk · 2026-08-10 · 约 79 分钟 · 原视频 ↗

本期速览 · Overview

一位物理学家探讨统计物理原理如何解释大脑的高效性，以及为什么在潜在空间预测优于在词元空间预测。

A physicist explores how principles from statistical physics explain the brain's efficiency and why predicting in latent space beats token space.

深度架构学习粗粒度变量，从而能从少量示例中产生创造力。
Deep architectures learn coarse-grained variables, enabling creativity from few examples.
层次化抽象克服维度灾难，使学习具有样本效率。
Hierarchical abstractions overcome the curse of dimensionality, making learning sample-efficient.
语言模型的缩放定律源于文本的相关性和熵。
Scaling laws for language models derive from text correlation and entropy.

深层架构具有构建粗粒度变量的隐式偏差，从而能够从少量示例中实现创造力。
Deep architectures have an implicit bias to build coarse-grained variables, enabling creativity from few examples.
世界具有层级结构，学习抽象是克服维度灾难的关键。
The world has a hierarchical structure, and learning abstractions is key to overcoming the curse of dimensionality.
在潜在空间中进行预测比在词元空间中进行预测更具样本效率。
Predicting in latent space is more sample-efficient than predicting in token space.
语言模型的缩放定律可以从文本的相关性结构和熵中推导出来。
Scaling laws for language models can be derived from the correlation structure and entropy of text.
物理学为理解机器学习提供了有用的框架，包括相变和普适性。
Physics provides a useful framework for understanding machine learning, including phase transitions and universality.

乔姆斯基的刺激贫乏论证对于深层架构而言是错误的；它们可以从少量示例中学会创造。
Chomsky's poverty of stimulus argument is incorrect for deep architectures; they can learn to be creative from few examples.
双重下降现象是一种堵塞相变，这是物理学中的一个概念。
The double descent phenomenon is a jamming transition, a concept from physics.
维度灾难并非通过流形假说解决；需要深层架构来发现层级抽象。
The curse of dimensionality is not solved by the manifold hypothesis; deep architectures are needed to discover hierarchical abstractions.
仅仅扩大当前模型的规模可能无法实现真正的科学创造力；我们需要超越下一个词预测的新能力。
Scaling up current models may not lead to true scientific creativity; we need new abilities beyond next-token prediction.
扩散模型和Transformer具有相似的样本复杂度；差异仅在于填充被遮蔽词元的顺序。
Diffusion models and transformers have similar sample complexity; the difference is only in the order of filling masked tokens.