NdLinear: Preserving Multi-Dimensional Structure for Parameter-Efficient Neural Networks

Abstract

NdLinear applies separate transformations along tensor dimensions instead of flattening every input into a single vector. The paper studies parameter efficiency and applications across model families, including low-rank adaptation, while examining limitations when interactions are not separable across dimensions.

Ravid Shwartz-Ziv

AI Researcher

AI researcher at Meta MSL with a background in information theory and computational neuroscience, working on world models, memory, and compression. Former Assistant Professor and Faculty Fellow at NYU, collaborating on research across academia and industry.