Calibrating feature maps for deep CNNs

Many performance improvement techniques calibrate the outputs of convolutional layers to improve the performance of convolutional neural networks, e.g., Squeeze-and-Excitation Networks (SENets). These techniques train the network to extract calibration weights from the input itself. However, these methods increase the complexity of the model in order to perform calibration. We propose an approach to calibrate the outputs of convolutional layers efficiently. Specifically, we propose an architectural block called Accuracy Booster, which calibrates the convolutional layer outputs channel-wise while introducing minimal extra parameters and computation. We experimentally show that our approach achieves higher performance than existing calibration methods over several datasets and architectures while introducing lesser parameters than them. We also generalize our proposed block to calibrate the channel, width, and height of the layer output in parallel. We empirically show that this type of composite calibration performs better than applying channel-wise calibration, spatial calibration, or both. We validate our approach on the CIFAR-10/100, CUB, ImageNet, and MS-COCO datasets for various tasks. The ResNet-50 architecture with the accuracy booster block performs comparably on the classification task to the ResNet-152 architecture, which has more than twice the number of parameters. We empirically show that our method generalizes well to other tasks such as object detection. We perform extensive ablation experiments to validate our approach.