Damian Wirth discusses treating trained model weights as input data for new neural networks, enabling faster generation and analysis of models.
要点 · TL;DR
将模型权重视为数据，可以分析和生成新模型，无需测试数据即可预测准确率等属性。 Treating model weights as data enables analysis and generation of new models, predicting properties like accuracy without test data.
扩展权重空间学习需要多样化的模型动物园，而不仅仅是更多模型，才能跨架构泛化。 Scaling weight space learning requires diverse model zoos, not just more models, to generalize across architectures.
从权重生成模型可以替代特定任务的预训练，节省计算资源，并实现超个性化。 Generating models from weights can replace pre-training for specific tasks, saving compute and enabling hyper-personalization.
核心观点 · Key points
训练好的模型权重可以被视为数据，从而能够分析和生成新模型。 Trained model weights can be treated as data, enabling analysis and generation of new models.
权重空间学习可以在没有测试数据的情况下预测模型属性，如准确率。 Weight space learning can predict model properties like accuracy without test data.
扩展权重空间学习需要多样化的模型库，而不仅仅是更多模型。 Scaling weight space learning requires diverse model zoos, not just more models.
从权重生成模型可以替代特定任务的预训练，节省算力。 Generating models from weights can replace pre-training for specific tasks, saving compute.
神经网络的基础模型可以为任何架构按需采样模型。 A foundation model of neural networks could sample on-demand models for any architecture.
反共识 · Contrarian takes
与其说数据枯竭，不如复用现有模型的权重作为新数据源。 Instead of running out of data, we should reuse weights of existing models as a new data source.
权重空间学习可以用更少的算力超越传统微调。 Weight space learning can outperform traditional fine-tuning with far less compute.
从权重生成模型可以比从头训练更高效，即使跨领域也是如此。 Generating models from weights can be more efficient than training from scratch, even across domains.
数据集嵌入可以在不泄露私有数据的情况下提示模型生成。 A dataset embedding can prompt model generation without revealing private data.
权重空间学习可以在单次前向传播中实现超个性化模型。 Weight space learning could enable hyper-personalized models in a single forward pass.
本期章节 · Chapters（共 15）
引言Introduction
基于权重的学习起源Origin of Weight-Based Learning
早期工作与灵感Early Work and Inspiration
初步构想与首篇论文Initial Idea and First Paper
扩展与合作Scaling up and collaboration
探索权重空间变换Exploring weight space transformations
与先前工作的联系Connection to prior work
在Hugging Face模型上的扩展与训练Scaling and Training on Hugging Face Models
恢复微调与神经网络基础模型的愿景Recovering Fine-Tuning and the Vision of a Foundation Model of Neural Networks
数据集提示Data Set Prompting
生成多模型与任务向量Generating Multiple Models and Task Vectors