Auto-sync: update nexus workspace
This commit is contained in:
22
wiki/concepts/Data-Distillation.md
Normal file
22
wiki/concepts/Data-Distillation.md
Normal file
@@ -0,0 +1,22 @@
|
||||
---
|
||||
title: "Data Distillation"
|
||||
type: concept
|
||||
tags: [distillation, model-compression, training, llm]
|
||||
aliases: [Data Distillation, 数据蒸馏, Knowledge Distillation]
|
||||
last_updated: 2025-12-20
|
||||
---
|
||||
|
||||
## Definition
|
||||
Data Distillation,数据蒸馏,利用高性能的大模型生成精简但有价值的数据,使一个小模型可以从中学习并逼近大模型的效果。
|
||||
|
||||
## Key Facts
|
||||
- 核心思想:用大模型作为"教师"(Teacher),生成高质量训练数据
|
||||
- 小模型(Student)从这些数据中学习
|
||||
- 目标:以更低成本达到接近大模型的效果
|
||||
- 是模型压缩和高效部署的重要技术手段
|
||||
|
||||
## Connections
|
||||
- [[Large Language Model]] ← 教师模型 ← [[Data Distillation]]
|
||||
|
||||
## Sources
|
||||
- [[大模型相关术语和框架总结|llm-mcp-prompt-rag-vllm-token-数据蒸馏]]
|
||||
Reference in New Issue
Block a user