Review

To Compress or Not to Compress--Self-Supervised Learning and Information Theory: A Review

We present a comprehensive review of self-supervised learning through the lens of information theory, introducing a unified framework that encompasses existing approaches and highlighting the interplay between compression and information preservation in deep neural networks.