TY - GEN
T1 - Leveraging filter correlations for deep model compression
AU - Singh, Pravendra
AU - Verma, Vinay Kumar
AU - Rai, Piyush
AU - Namboodiri, Vinay P.
PY - 2020/5/14
Y1 - 2020/5/14
N2 - We present a filter correlation based model compression approach for deep convolutional neural networks. Our approach iteratively identifies pairs of filters with the largest pairwise correlations and drops one of the filters from each such pair. However, instead of discarding one of the filters from each such pair naïvely, the model is re-optimized to make the filters in these pairs maximally correlated, so that discarding one of the filters from the pair results in minimal information loss. Moreover, after discarding the filters in each round, we further finetune the model to recover from the potential small loss incurred by the compression. We evaluate our proposed approach using a comprehensive set of experiments and ablation studies. Our compression method yields state-of-the-art FLOPs compression rates on various benchmarks, such as LeNet-5, VGG-16, and ResNet-50, 56, while still achieving excellent predictive performance for tasks such as object detection on benchmark datasets.
AB - We present a filter correlation based model compression approach for deep convolutional neural networks. Our approach iteratively identifies pairs of filters with the largest pairwise correlations and drops one of the filters from each such pair. However, instead of discarding one of the filters from each such pair naïvely, the model is re-optimized to make the filters in these pairs maximally correlated, so that discarding one of the filters from the pair results in minimal information loss. Moreover, after discarding the filters in each round, we further finetune the model to recover from the potential small loss incurred by the compression. We evaluate our proposed approach using a comprehensive set of experiments and ablation studies. Our compression method yields state-of-the-art FLOPs compression rates on various benchmarks, such as LeNet-5, VGG-16, and ResNet-50, 56, while still achieving excellent predictive performance for tasks such as object detection on benchmark datasets.
UR - http://www.scopus.com/inward/record.url?scp=85085507548&partnerID=8YFLogxK
U2 - 10.1109/WACV45572.2020.9093331
DO - 10.1109/WACV45572.2020.9093331
M3 - Chapter in a published conference proceeding
AN - SCOPUS:85085507548
T3 - Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020
SP - 824
EP - 833
BT - Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020
PB - IEEE
T2 - 2020 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2020
Y2 - 1 March 2020 through 5 March 2020
ER -