A high-level overview of various machine learning algorithms, unsupervised learning, and the role of probability in the hypothesis class, loss function, and optimization framework.
要点 · TL;DR
监督学习=假设类+损失+优化；无监督学习旨在重构输入。 Supervised learning = hypothesis class + loss + optimization; unsupervised aims to reconstruct input.
概率模型通过最大似然估计为损失函数提供依据。 Probabilistic models justify loss functions via maximum likelihood estimation.
正确评估需要划分训练集、验证集和测试集。 Proper evaluation requires separate training, validation, and test sets.
核心观点 · Key points
监督学习算法可被框架化为假设类、损失函数和优化方法。 Supervised learning algorithms can be framed as hypothesis class, loss function, and optimization method.
无监督学习旨在重构输入数据，需要限制假设类以避免平凡解。 Unsupervised learning aims to reconstruct input data, requiring restricted hypothesis classes to avoid trivial solutions.
概率模型通过最大似然估计为平方损失和逻辑损失等损失函数提供了合理性。 Probabilistic models justify loss functions like squared loss and logistic loss via maximum likelihood estimation.
正确评估需要将数据分为训练集、验证集和测试集，以避免过拟合。 Proper evaluation requires splitting data into training, validation, and testing sets to avoid overfitting.
核方法在参数上是线性的，但需要存储所有训练数据以进行预测。 Kernel methods are linear in parameters but require storing all training data for predictions.
反共识 · Contrarian takes
像 k 近邻这样的非参数方法没有训练阶段，但需要存储全部数据。 Nonparametric methods like k-nearest neighbors have no training phase but require full data storage.
决策树通过贪心方式训练，而非精确优化，但在集成中效果显著。 Decision trees are trained greedily, not via exact optimization, yet are effective in ensembles.
主成分分析（PCA）尽管是非凸问题，但可通过特征值分解精确求解。 PCA can be solved exactly via eigenvalue decomposition despite being a non-convex problem.
使用验证误差选择超参数并将其报告为测试误差是一个常见错误。 Using validation error to select hyperparameters and then reporting it as test error is a common mistake.
神经网络无法保证找到最优参数，但仍取得了重大成功。 Neural networks have no guarantee of finding optimal parameters, yet they achieve major successes.
本期章节 · Chapters（共 21）
0. 引言与概述Introduction and Overview
1. 框架回顾：假设类、损失函数、优化Framework Recap: Hypothesis Class, Loss Function, Optimization
2. 核方法概述Kernel Methods Overview
3. 核方法与正则化Kernel Methods and Regularization
4. 最近邻方法Nearest Neighbor Methods
5. 神经网络Neural Networks
6. 决策树Decision Trees
7. 决策树与假设类Decision Trees and Hypothesis Class
8. 决策树中的损失函数与优化Loss Function and Optimization in Decision Trees
9. 集成方法：随机森林与提升Ensemble Methods: Random Forests and Boosting
10. 无监督学习概述Unsupervised Learning Overview
11. 无监督学习设置Unsupervised Learning Setup
12. 为何不直接用恒等函数？Why Not Just Identity Function?
13. K 均值示例K-Means Example
14. K 均值直觉与损失函数K-means intuition and loss function