Context extraction module for deep convolutional neural networks

Pravendra Singh, Pratik Mazumder, Vinay P. Namboodiri

Research output: Contribution to journalArticlepeer-review

7 Citations (SciVal)


Convolutional layers convolve the input feature maps to generate valuable output features, and they help deep learning methods significantly in solving complex problems. In order to tackle problems efficiently, deep learning solutions should ensure that the parameters of the model do not increase significantly with the complexity of the problem. Pointwise convolutions are primarily used for parameter reduction in many deep learning architectures. They are convolutional filters of kernel size 1×1. The pointwise convolution, however, ignores the spatial information around the points it is processing. This design is by choice, in order to reduce the overall parameters and computations. However, we hypothesize that this shortcoming of pointwise convolution has a significant impact on network performance. We propose a novel alternative design for pointwise convolution, which uses spatial information from the input efficiently. Our approach extracts spatial context information from the input at two scales and further refines the extracted context based on the channel importance. Finally, we add the refined context to the output of the pointwise convolution. This is the first work that improves pointwise convolution by incorporating context information. Our design significantly improves the performance of the networks without substantially increasing the number of parameters and computations. We perform experiments on coarse/fine-grained image classification, few-shot fine-grained classification, and on object detection. We further perform various ablation experiments to validate the significance of the different components used in our design. Lastly, we show experimentally that our proposed technique can be combined with existing state-of-the-art network performance improvement approaches to further improve the network performance.

Original languageEnglish
Article number108284
JournalPattern Recognition
Early online date30 Aug 2021
Publication statusPublished - 28 Feb 2022


  • Contextual pointwise convolution
  • Convolutional neural network (CNN)
  • Deep learning
  • Image classification

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence


Dive into the research topics of 'Context extraction module for deep convolutional neural networks'. Together they form a unique fingerprint.

Cite this