Transformers

UAT-LITE: Inference-Time Uncertainty-Aware Attention for Pretrained Transformers

Making pretrained transformer attention sensitive to uncertainty during inference.

Seq-VCR: Preventing Collapse in Intermediate Transformer Representations for Enhanced Reasoning

Regularizing intermediate representations to reduce collapse and support sequential reasoning.