Sujay Sanghavi | Ravid Shwartz-Zivhttps://www.ravid-shwartz-ziv.com/author/sujay-sanghavi/Sujay SanghaviSource Themes Academic (https://sourcethemes.com/academic/)en-usMon, 16 Feb 2026 00:00:00 +0000https://www.ravid-shwartz-ziv.com/img/ravid-shwartz-ziv-social-v2.pngSujay Sanghavihttps://www.ravid-shwartz-ziv.com/author/sujay-sanghavi/When Attention Collapses: How Degenerate Layers in LLMs Enable Smaller, Stronger Modelshttps://www.ravid-shwartz-ziv.com/publication/inuit/Mon, 16 Feb 2026 00:00:00 +0000https://www.ravid-shwartz-ziv.com/publication/inuit/<p>Previously titled <strong>Inheritune: Training Smaller Yet More Attentive Language Models</strong>. First posted on arXiv on April 12, 2024; the journal version was published in TMLR in February 2026. The arXiv version was updated on February 16, 2026.</p>