• A
  • A
  • A
  • ABC
  • ABC
  • ABC
  • А
  • А
  • А
  • А
  • А
Regular version of the site
Student
Title
Supervisor
Faculty
Educational Programme
Final Grade
Year of Graduation
Valerii Kharitonov
Bayesian Pruning of Neural Networks
Data Science
(Master’s programme)
2018
We study the empirical Bayes approach to hyperparameter optimization and model selection in Bayesian neural networks. We explore its automatic relevance determination and pruning effect when Gaussian prior and approximate posterior for neural network weights are used. The connection is drawn between Sparse Variational Dropout and Gaussian Automatic Relevance Determination objectives. Based on the empirical Bayes procedure, we propose a novel type of prior that encourages structured sparsity and allows for compression and speeding up of modern convolutional deep neural network architectures. We provide experimental results showing that the proposed structured pruning procedure removes most of the filters or neurons in convolutional and dense layers in LeNet5 and VGG-like convolutional neural networks with almost no drop in accuracy of classification on MNIST and CIFAR-10 datasets. The new approach outperforms previously reported results in terms of sparsity levels in convolutional layers.

Student Theses at HSE must be completed in accordance with the University Rules and regulations specified by each educational programme.

Summaries of all theses must be published and made freely available on the HSE website.

The full text of a thesis can be published in open access on the HSE website only if the authoring student (copyright holder) agrees, or, if the thesis was written by a team of students, if all the co-authors (copyright holders) agree. After a thesis is published on the HSE website, it obtains the status of an online publication.

Student theses are objects of copyright and their use is subject to limitations in accordance with the Russian Federation’s law on intellectual property.

In the event that a thesis is quoted or otherwise used, reference to the author’s name and the source of quotation is required.

Search all student theses