AI Researcher Meta MSL

Ravid Shwartz-Ziv

I’m an AI researcher at Meta Superintelligence Labs (MSL), focused on world models, memory, and compression.

My work connects representation learning, continual learning, and AI agents to build systems that predict, retain and update knowledge, adapt over time, and use computation efficiently.

Previously, I was an Assistant Professor and Faculty Fellow at NYU’s Center for Data Science. I completed my Ph.D. with Tali Tishby and postdoctoral research with Yann LeCun. My background is in information theory and computational neuroscience; my industry experience spans Wand AI, Intel, and Google AI.

Explore my work Share an idea

Research across academia and industry

World models Memory Compression

Three connected directions

Research focus

Three connected directions for building AI systems that understand, remember, and adapt.

Understand & predict

World Models

Learning predictive representations of how environments change, and using them for reasoning, planning, and action.

Explore this research area

Retain & adapt

Memory & Continual Learning

Helping models preserve useful knowledge, update it without destructive interference, and improve through experience.

Explore this research area

Build efficiently

Compression & Efficient AI

Reducing memory and computation while preserving the representations and capabilities that matter for a task.

Explore this research area

Methods, tools, systems

Research into practice

Research methods, open-source tools, and practical AI systems.

World models & JEPA

S-JEPA

Learning speech representations through masked prediction with soft clustering targets. This work explores predictive representations relevant to world models.

Explore the code Read the paper

Compression & efficiency

Inheritune

Building smaller language models by reusing useful layers and reducing the redundancy caused by attention collapse.

Explore the code Read the paper

Evaluation

LiveBench

An evolving benchmark that evaluates language models using recent questions, objective scoring, and a focus on reducing test contamination.

View the benchmark Read the paper

AI agents

Minitap

A mobile agent system that separates planning, execution, and verification, studied through task completion on AndroidWorld.

Explore the code Read the paper

Research collaborations

I enjoy working with researchers and teams across academia and industry on shared research questions and practical AI systems. These collaborations range from joint research projects to exchanging ideas and offering guidance as new directions take shape.

Discuss a collaboration

Selected Research

Layer by Layer: Uncovering Hidden Representations in Language Models

Finding useful embeddings inside language models. We study why intermediate layers can outperform the final layer and how to identify them using information, geometry, and downstream evaluation.

ICML 2025 (Oral)

Oscar Skean, Md Rifat Arefin, Dan Zhao, Niket Nikul Patel, Jalal Naghiyev, Yann LeCun, Ravid Shwartz-Ziv

PDF Code arXiv Proceedings Oral presentation

Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs

A decoding method that scales its token cutoff with the model’s confidence, giving a simple way to control sampling as temperature changes.

ICLR 2025 (Oral)

Minh Nhat Nguyen, Andrew Baker, Clement Neo, Allen Roush, Andreas Kirsch, Ravid Shwartz-Ziv

PDF Code arXiv Proceedings

Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs

S-JEPA: Soft Clustering Anchors for Self-Supervised Speech Representation Learning

Learning speech representations with soft predictive targets, preserving acoustic ambiguity while avoiding repeated offline reclustering.

arXiv, 2026

Georgios Ioannides, Adrian Kieback, Judah Goldfeder, Linsey Pang, Aman Chadha, Aaron Elkins, Yann LeCun, Ravid Shwartz-Ziv

PDF Code arXiv

On Training in Imagination

Training agents in learned world models: how model errors and rollout budgets shape policy learning.

arXiv, 2026

Nadav Timor, Ravid Shwartz-Ziv, Micah Goldblum, Yann LeCun, David Harel

PDF arXiv

You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations

Compressing an LLM for the task it actually needs to perform, allocating precision to the layers that matter.

ICML 2026 AdaptFM Workshop

Amit LeVi, Raz Lapid, Rom Himelstein, Chaim Baskin, Ravid Shwartz-Ziv, Avi Mendelson

PDF arXiv

Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coin

Connecting attention sinks to representation compression to explain how information changes across LLM layers.

ICLR 2026

Enrique Queipo-de-Llano, Álvaro Arroyo, Federico Barbero, Xiaowen Dong, Michael Bronstein, Yann LeCun, Ravid Shwartz-Ziv

PDF arXiv Publication

See all publications

Recent Publications

Quickly discover relevant content by filtering publications.

Don't Repeat Yourself: Stopping Verbatim Loops at Sampling Time

Reducing verbatim loops during generation by penalizing tokens that continue a previously seen sequence.

arXiv, 2026

Philipp Emanuel Weidmann, Allen Roush, Judah Goldfeder, Sanjay Basu, Ravid Shwartz-Ziv

PDF arXiv

XTC: Head-Aware Sampling by Excluding Top Choices

Encouraging varied text by occasionally excluding dominant choices when several plausible next tokens are available.

arXiv, 2026

Philipp Emanuel Weidmann, Allen Roush, Judah Goldfeder, Sanjay Basu, Ravid Shwartz-Ziv

PDF arXiv

HP-JEPA: Hierarchical Partitioning for Multi-Resolution Graph Joint-Embedding Predictive Learning

Learning graph representations at multiple scales through latent prediction over coarse and fine partitions.

arXiv, 2026

Ruichen Xu, Jingxiang Qu, Wenhan Gao, Jiaxing Zhang, Linsey Pang, Ravid Shwartz-Ziv, Yann LeCun, Yuefan Deng

PDF arXiv

See all publications

The Information Bottleneck

Writing and conversations about AI research

On The Information Bottleneck, I write about AI research and the details that matter when building systems. I also co-host the podcast with Allen Roush, talking with researchers about their ideas and what they mean in practice.

The name comes from a simple idea: compressing information while preserving what matters for a task. Our conversations range from memory and reasoning to AI agents and the foundations of learning.

Selected writing

Editing a Compressed Memory — How linear attention stores context in a fixed-size state, and how the delta rule changes what it remembers. June 2026.
Speculative decoding, from zero to DSpark — From drafting and verification to the scheduling decisions that make inference faster in a serving system. July 2026.

The podcast

Apple Podcasts · Spotify

Experience

Selected research appointments

AI Researcher

Meta Superintelligence Labs (MSL)

August 2025 – Present

Research on world models, memory, compression, and continual learning, with an emphasis on practical capabilities for efficient language models and AI agents.

Assistant Professor and Faculty Fellow

NYU, Center for Data Science

Former appointment · started September 2021 New York, NY

Led research initiatives in LLMs, focusing on model efficiency, compression techniques, and novel benchmarking frameworks
Studied approaches for analyzing LLM representations and information flow
Conducted research in representation learning through information-theoretic lens
Developed and taught graduate-level courses in Advanced Machine Learning and Deep Learning

Senior AI Researcher & Team Lead

Wand AI

January 2023 – August 2025 New York, NY

Developed LLM personalization techniques
Researched efficient adaptation methods for LLMs
Directed end-to-end development from research prototypes to production

Earlier industry and research roles

Senior AI & Data Science Researcher

Intel

Jan 2020 – Dec 2023 New York, NY

Led development of AI algorithms focusing on LLMs and RAG systems
Optimized validation processes for automated code validation
Collaborated with cross-functional teams on tabular data solutions

Research Student

Google AI

Jun 2019 – May 2020 Mountain View, CA

Developed an information-theoretic framework for infinitely-wide neural networks
Created efficient data compression algorithms leveraging information theory

AI & Data Science Researcher

Intel

Feb 2013 – May 2019 Petah-Tikva, Israel

Developed computer vision solutions for GPU defect detection
Created ML-based automated testing frameworks
Designed sensor-based ML systems for healthcare monitoring

Algorithm and Web Developer

Wikipedia

Jan 2010 – Jan 2013 Israel

Developed machine learning projects, including OCR system and copyright detection tool
Enhanced user experience through creation of editing gadgets
Contributed to projects promoting public knowledge accessibility

Talks archive All publications

Education

Ph.D. in Computational Neuroscience, Hebrew University of Jerusalem (2021)
Advisor: Tali Tishby

B.Sc. in Computer Science and Computational Biology, Hebrew University of Jerusalem (2014)

Ravid Shwartz-Ziv

Research focus

World Models

Memory & Continual Learning

Compression & Efficient AI

Research into practice

S-JEPA

Inheritune

LiveBench

Minitap

Research collaborations

Selected Research

Recent Publications

The Information Bottleneck

Selected writing

The podcast

Experience

AI Researcher

Assistant Professor and Faculty Fellow

Senior AI Researcher & Team Lead

Senior AI & Data Science Researcher

Research Student

AI & Data Science Researcher

Algorithm and Web Developer

Education

Let’s talk