Leveraging filter correlations for deep model compression

Pravendra Singh, Vinay Kumar Verma, Piyush Rai, Vinay P. Namboodiri

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

We present a filter correlation based model compression approach for deep convolutional neural networks. Our approach iteratively identifies pairs of filters with the largest pairwise correlations and drops one of the filters from each such pair. However, instead of discarding one of the filters from each such pair naïvely, the model is re-optimized to make the filters in these pairs maximally correlated, so that discarding one of the filters from the pair results in minimal information loss. Moreover, after discarding the filters in each round, we further finetune the model to recover from the potential small loss incurred by the compression. We evaluate our proposed approach using a comprehensive set of experiments and ablation studies. Our compression method yields state-of-the-art FLOPs compression rates on various benchmarks, such as LeNet-5, VGG-16, and ResNet-50, 56, while still achieving excellent predictive performance for tasks such as object detection on benchmark datasets.

Original languageEnglish
Title of host publicationProceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020
PublisherIEEE
Pages824-833
Number of pages10
ISBN (Electronic)9781728165530
DOIs
Publication statusPublished - Mar 2020
Event2020 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2020 - Snowmass Village, USA United States
Duration: 1 Mar 20205 Mar 2020

Publication series

NameProceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020

Conference

Conference2020 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2020
CountryUSA United States
CitySnowmass Village
Period1/03/205/03/20

ASJC Scopus subject areas

  • Computer Science Applications
  • Computer Vision and Pattern Recognition

Cite this

Singh, P., Verma, V. K., Rai, P., & Namboodiri, V. P. (2020). Leveraging filter correlations for deep model compression. In Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020 (pp. 824-833). [9093331] (Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020). IEEE. https://doi.org/10.1109/WACV45572.2020.9093331