UAT-LITE: Inference-Time Uncertainty-Aware Attention for Pretrained Transformers

Abstract

UAT-LITE uses stochastic forward passes to estimate token-level uncertainty and adjust attention in pretrained classifiers. The framework studies calibration, selective prediction, and uncertainty across layers without updating pretrained model weights.

Publication
arXiv preprint
Ravid Shwartz-Ziv
Ravid Shwartz-Ziv
AI Researcher

AI researcher at Meta MSL with a background in information theory and computational neuroscience, working on world models, memory, and compression. Former Assistant Professor and Faculty Fellow at NYU, collaborating on research across academia and industry.

Related