Transfer Learning

When Attention Collapses: How Degenerate Layers in LLMs Enable Smaller, Stronger Models

Inheritune builds smaller language models by reusing useful layers and reducing attention collapse, while retaining or improving performance.

Fine-Tuning with Uncertainty-Aware Priors Makes Vision and Language Foundation Models More Reliable

Improving the reliability of fine-tuned foundation models with priors that account for uncertainty.