Ravid Shwartz-Ziv

Ravid Shwartz-Ziv

Research scientist & PhD Student

Intel and the Hebrew University of Jerusalem

Biography

I’m a Ph.D. candidate under the supervision of Prof Naftali Tishby and Prof Haim Sompolinsky at the Hebrew University of Jerusalem.

In my Ph.D., I focused on the connection between deep neural networks (DNNs) and information theory. I tried to develop a deeper understating of DNNs based on information theory and to implement it over large scale problems.

In parallel to my studies, I work as a researcher at the A.I. & data science research team of Intel’s Advanced Analytics group. There, I am involved in several projects. Mainly, development of deep learning, computer vision, and sensory data solutions for healthcare, manufacturing, and marketing, for both internal and external uses.

Recently I had the opportunity to work as a research student at Google Brain, CA, USA. In this position, I explored the generalization ability of DNNs using information theory tools.

In the past, I was also involved in developing some projects for Wikipedia.

In my free, I volunteer as a developer at The Public Knowledge Workshop.

My research is supported by a Google PhD Fellowship.

And I love basketball :)

Interests

  • Artificial Intelligence
  • Computational Neuroscience
  • Information Theory

Education

  • PhD in Computer Science and Neuroscience, 2020

    The Hebrew University of Jerusalem

  • MSc in Computer Science and Neuroscience, 2016

    The Hebrew University of Jerusalem

  • BSc in Computer Science and Bioinformatics, 2014

    The Hebrew University of Jerusalem

Experience

 
 
 
 
 

Research Student

Google AI, Host: Dr. Alex Alemi

Jun 2019 – May 2020 Mountain View, CA, USA
Exploration of generalization by information quantities for infinitely-wide neural networks.
 
 
 
 
 

Graduate Researcher

Advisor: Professor Naftali Tishby - The Hebrew University of Jerusalem

Jan 2016 – Present Israel

Empirical and theoretical study of DNNs based on information-theoretical principles.

  • Development of a deeper understating of DNNs based on information theory.
  • Devise large scale implementation algorithms for the information bottleneck theory.
 
 
 
 
 

Graduate Researcher

Advisor: Professor Haim Sompolinsky - The Hebrew University of Jerusalem

Jan 2016 – Present Israel
Development of models for perceptual and transfer learning in DNNs, which are biologically plausible.
 
 
 
 
 

Senior AI and Data Science Researcher

Intel

Feb 2013 – Present Israel

Developing novel deep learning, computer vision and sensory data solutions for healthcare, manufacturing, sales, and marketing for both internal and external usage. Selected Projects

  • Automated testing of graphics units with video anomaly detection.
  • Optimization of the validation process using images transfer learning.
  • Healthcare solutions using physical and virtual sensors.
  • Gait recognition for smartphones using sensors fusion.
 
 
 
 
 

Research Assistant

Advisor: Professor Leo Joskowicz - The Hebrew University of Jerusalem

Feb 2012 – Feb 2014 Israel
Development of image segmentation algorithms using DNNs used for extracting medical indicators to detect abnormality in embryos.

Recent Publications

Quickly discover relevant content by filtering publications.

The Dual Information Bottleneck

A new framework, which resolves some of the known drawbacks of the Information Bottleneck. We provide a theoretical analysis of the framework, finding the structure of its solutions and present a novel variational formulation for DNNs.
The Dual Information Bottleneck

Information in Infinite Ensembles of Infinitely-Wide Neural Networks

Study the generalization properties of infinite ensembles of infinitely-wide neural networks. We report analytical and empirical investigations in the search for signals that correlate with generalization.
Information in Infinite Ensembles of Infinitely-Wide Neural Networks

Neural Correlates of Learning Pure Tones or Natural Sounds in the Auditory Cortex

Analysing perceptual learning of pure tones in the auditory cortex. Using a novel computational model, we show that overrepresentation of the learned tones does not improve along the training.
Neural Correlates of Learning Pure Tones or Natural Sounds in the Auditory Cortex

Attentioned Convolutional LSTM Inpaintingv Network for Anomaly Detection in Videos

A semi supervised model for detecting anomalies in videos inspiredby the Video Pixel Network. We extend the Convolutional LSTM video encoder of the VPN with a novel convolutional based attention. This approach could be a component in applications requiring visual common sense.
Attentioned Convolutional LSTM Inpaintingv Network for Anomaly Detection in Videos

Sequence Modeling Using a Memory Controller Extension for LSTM

We extend the standard LSTM architecture by augmenting it with an additional gate which produces a memory control vector signal. This vector is fed back to the LSTM instead of the original output prediction. By decoupling the LSTM prediction from its role as a memory controller we allow each output to specialize in its own task.
Sequence Modeling Using a Memory Controller Extension for LSTM

Talks

Information in Infinite Ensembles of Infinitely-Wide Neural Networks

Finding generalization signals using information for infinitely-wide neural networks.
Information in Infinite Ensembles of Infinitely-Wide Neural Networks

Representation Compression in Deep Neural Network

An information theoretic viewpoint on the behavior of deep networks optimization processes and their generalization abilities by the information plane and how compression can help.
Representation Compression in Deep Neural Network

On the Information Theory of Deep Neural Networks

Understanding Deep Neural Networks with the information bottleneck principle.
On the Information Theory of Deep Neural Networks

Open the Black Box of Deep Neural Networks

Where is the information in deep neural networks? trying to find it by looking on the information plane.
Open the Black Box of Deep Neural Networks

Analysis and Theory of Perceptual Learning in Auditory Cortex

Analysing perceptual learning of pure tones in the auditory cortex. Using a novel computational model, we show that overrepresentation of the learned tones does not necessarily improve along the training
Analysis and Theory of Perceptual Learning in Auditory Cortex

Contact