In the last years, deep neural networks have revolutionized machine learning tasks. However, the design of deep neural network architectures is still based on try-and-error procedures, and they are usually complex models with high computational cost. This is the reason behind the efforts that are made in the deep learning community to create small and compact models with comparable accuracy to the current deep neural networks. In literature, different methods to reach this goal are presented; among them, techniques based on low rank factorization are used in order to compress pre trained models with the aim to provide a more compact version of them without losing their effectiveness. Despite their promising results, these techniques produce auxiliary structures between network layers; this work shows that is possible to overcome the need for such elements by using simple regularization techniques. We tested our approach on the VGG16 model obtaining a four times faster reduction without loss in accuracy and avoiding supplementary structures between the network layers.
Linear Regularized Compression of Deep Convolutional Neural Networks / C. Ceruti, P. Campadelli, E. Casiraghi (LECTURE NOTES IN COMPUTER SCIENCE). - In: Image Analysis and Processing : ICIAP / [a cura di] S. Battiato, G. Gallo, R. Schettini, F. Stanco. - [s.l] : Springer, 2017 Sep 13. - ISBN 9783319685601. - pp. 244-253 (( Intervento presentato al 19. convegno ICIAP tenutosi a Catania nel 2017 [10.1007/978-3-319-68560-1_22].
Linear Regularized Compression of Deep Convolutional Neural Networks
C. CerutiPrimo
;P. CampadelliSecondo
;E. CasiraghiUltimo
2017
Abstract
In the last years, deep neural networks have revolutionized machine learning tasks. However, the design of deep neural network architectures is still based on try-and-error procedures, and they are usually complex models with high computational cost. This is the reason behind the efforts that are made in the deep learning community to create small and compact models with comparable accuracy to the current deep neural networks. In literature, different methods to reach this goal are presented; among them, techniques based on low rank factorization are used in order to compress pre trained models with the aim to provide a more compact version of them without losing their effectiveness. Despite their promising results, these techniques produce auxiliary structures between network layers; this work shows that is possible to overcome the need for such elements by using simple regularization techniques. We tested our approach on the VGG16 model obtaining a four times faster reduction without loss in accuracy and avoiding supplementary structures between the network layers.File | Dimensione | Formato | |
---|---|---|---|
iciap2017.pdf
accesso riservato
Tipologia:
Publisher's version/PDF
Dimensione
219.69 kB
Formato
Adobe PDF
|
219.69 kB | Adobe PDF | Visualizza/Apri Richiedi una copia |
Pubblicazioni consigliate
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.