Fine-Tuning with Uncertainty-Aware Priors Makes Vision and Language Foundation Models More Reliable
Abstract
We construct data-dependent priors that encourage uncertainty on inputs unlike the fine-tuning data. Approximate inference with these priors is evaluated through calibration, selective prediction, and semantic distribution shifts in vision and language tasks.
Type
Publication
Proceedings of the 28th International Conference on Artificial Intelligence and Statistics
Ravid Shwartz-Ziv
AI Researcher
AI researcher at Meta MSL with a background in information theory and computational neuroscience, working on world models, memory, and compression. Former Assistant Professor and Faculty Fellow at NYU, collaborating on research across academia and industry.