Self-Supervised Learning

HP-JEPA: Hierarchical Partitioning for Multi-Resolution Graph Joint-Embedding Predictive Learning

Learning graph representations at multiple scales through latent prediction over coarse and fine partitions.

S-JEPA: Soft Clustering Anchors for Self-Supervised Speech Representation Learning

Learning speech representations with soft predictive targets, preserving acoustic ambiguity while avoiding repeated offline reclustering.

An Information Theory Perspective on Variance-Invariance-Covariance Regularization

We provide an information-theoretic analysis of VICReg, deriving theoretical foundations for deterministic networks and introducing new SSL methods based on these insights.

To Compress or Not to Compress--Self-Supervised Learning and Information Theory: A Review

We present a comprehensive review of self-supervised learning through the lens of information theory, introducing a unified framework that encompasses existing approaches and highlighting the interplay between compression and information preservation in deep neural networks.