Representation Learning

Comparing how language models and humans balance compact representations with semantic detail.

Reducing parameter costs with linear transformations that preserve the structure of multidimensional inputs.

Regularizing intermediate representations to reduce collapse and support sequential reasoning.

We provide an information-theoretic analysis of VICReg, deriving theoretical foundations for deterministic networks and introducing new SSL methods based on these insights.