Compression of convolutional neural networks using Tucker Decomposition
Nowadays, deep neural networks are being introduced in mobile devices where memory space and computation speed is limited. Therefore, the need for more com- pression in those networks has increased. In this work, we explore a way to com- press the convolutional neural networks by reducing the number...
| Autor: | |
|---|---|
| Tipo de recurso: | tesis de maestría |
| Fecha de publicación: | 2017 |
| País: | España |
| Institución: | Universitat Politècnica de Catalunya (UPC) |
| Repositorio: | UPCommons. Portal del coneixement obert de la UPC |
| Idioma: | inglés |
| OAI Identifier: | oai:upcommons.upc.edu:2117/106019 |
| Acceso en línea: | https://hdl.handle.net/2117/106019 |
| Access Level: | acceso abierto |
| Palabra clave: | Neural networks (Computer science) CNN tucker Xarxes neuronals (Informàtica) Àrees temàtiques de la UPC::Enginyeria de la telecomunicació::Telemàtica i xarxes d'ordinadors |
| Sumario: | Nowadays, deep neural networks are being introduced in mobile devices where memory space and computation speed is limited. Therefore, the need for more com- pression in those networks has increased. In this work, we explore a way to com- press the convolutional neural networks by reducing the number of weights in their convolutional layers. This method consists on the Tucker decomposition of tensors introduced on the kernels of the convolutional layers. Two different architectures, one with three convolutional layers and another with five, are the base of our study. We employ the Tucker decomposition in different forms, with more or less compression. We compute their compression rates versus the accuracy and loss (training and test) for the CIFAR-10 dataset, in order to see how good the performance of the compressed models is with respect to the original. The decomposition achieves its best results when compressing partially each kernel layer, up to 70%. Finally, we visualize the probability density function of the filters, contrasting its shape in function of the compression. From those plots, we conclude that, not only we are able to reduce the number of parameters, but also the number of bits needed to store them if we use entropy encoding. |
|---|