2 | Ravid Shwartz-Ziv

When Attention Collapses: How Degenerate Layers in LLMs Enable Smaller, Stronger Models

Mon, 16 Feb 2026 00:00:00 +0000

Previously titled Inheritune: Training Smaller Yet More Attentive Language Models. First posted on arXiv on April 12, 2024; the journal version was published in TMLR in February 2026. The arXiv version was updated on February 16, 2026.

To Compress or Not to Compress--Self-Supervised Learning and Information Theory: A Review

Wed, 19 Apr 2023 00:00:00 +0000

Tabular Data: Deep Learning is Not All You Need

Sat, 01 Jan 2022 00:00:00 +0000

Neural Correlates of Learning Pure Tones or Natural Sounds in the Auditory Cortex

Wed, 01 Jan 2020 00:00:00 +0000

Sequence Modeling Using a Memory Controller Extension for LSTM

Fri, 01 Dec 2017 00:00:00 +0000

Opening the Black Box of Deep Neural Networks via Information

Sat, 01 Jul 2017 00:00:00 +0000