Information in Infinite Ensembles of Infinitely-Wide Neural Networks

Image credit: Unsplash

Abstract

In this work, we study the generalization properties of infinite ensembles of infinitely-wide neural networks. Amazingly, this model family admits tractable calculations for many information-theoretic quantities. We report analytical and empirical investigations in the search for signals that correlate with generalization.

Publication
Proceedings of the Symposium on Advances in Approximate Bayesian Inference
Ravid Shwartz-Ziv
Ravid Shwartz-Ziv
Research scientist & PhD Student

Related