PROOF-Gen: From Optimized Knowledge to Higher Distillation
Supervised fine-tuning on teacher-generated trajectories is the usual first stage for distilling tool-calling capabilities into deployable fashions. Publish-training pipelines that ...
Tag: Distillation
Locking Pretrained Weights through Deep Low-Rank Residual Distillation
The standard of open-weight language fashions has dramatically improved lately. Sharing weights drastically facilitates mannequin adoption by enabling their use ...
Unmasking On-Coverage Distillation: The place It Helps, The place It Hurts, and Why
On-policy distillation gives dense, per-token supervision for coaching reasoning fashions; nevertheless, it stays unclear underneath which situations this sign is ...
![[2606.22600] On the Place Bias of On-Coverage Distillation](http://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png "[2606.22600] On the Place Bias of On-Coverage Distillation"){kind=logo}
[2606.22600] On the Place Bias of On-Coverage Distillation
View a PDF of the paper titled On the Place Bias of On-Coverage Distillation, by Yan Xie and 4 different ...
The world's leading source for AI research, emerging technology, and the people building the future. Independent, rigorous, and always ahead.