Representation Learning

Ravid Shwartz Ziv's research on world models, predictive representations, learned dynamics, and training agents in imagination.

Learning graph representations at multiple scales through latent prediction over coarse and fine partitions.

Learning speech representations with soft predictive targets, preserving acoustic ambiguity while avoiding repeated offline reclustering.

An earlier speech JEPA study using fixed soft clustering targets to stabilize representation learning.

Connecting JEPA representations to speech compression through density-adaptive attention and compact audio tokens.

Compressing an LLM for the task it actually needs to perform, allocating precision to the layers that matter.

Using chess and Chess960, we investigate how strategic concepts appear across model layers and how stronger play can diverge from human conceptual understanding.

Connecting attention sinks to representation compression to explain how information changes across LLM layers.

Finding useful embeddings inside language models. We study why intermediate layers can outperform the final layer and how to identify them using information, geometry, and downstream evaluation.

Studying which layers support mathematical reasoning and how their roles persist after post-training.