Uncertainty Class Activation Map (U-CAM) using Gradient Certainty method

Badri Patro, Mayank Lunayach, Vinay Namboodiri

Research output: Contribution to journalArticlepeer-review

165 Downloads (Pure)

Abstract

Understanding and explaining deep learning models is an imperative task. Towards this, we propose a method that obtains gradient-based certainty estimates that also provide visual attention maps. Particularly, we solve for visual question answering task. We incorporate modern probabilistic deep learning methods that we further improve by using the gradients for these estimates. These have two-fold benefits: a) improvement in obtaining the certainty estimates that correlate better with misclassified samples and b) improved attention maps that provide state-of-the-art results in terms of correlation with human attention regions. The improved attention maps result in consistent improvement for various methods for visual question answering. Therefore, the proposed technique can be thought of as a tool for obtaining improved certainty estimates and explanations for deep learning models. We provide detailed empirical analysis for the visual question answering task on all standard benchmarks and comparison with state of the art methods.
Original languageEnglish
Article number9316944
Pages (from-to)1910-1924
Number of pages15
JournalIEEE Transactions on Image Processing
Volume30
Early online date8 Jan 2021
DOIs
Publication statusPublished - 31 Dec 2021

Keywords

  • Aleatoric
  • Bayesian model
  • CNN
  • LSTM
  • Uncertainty
  • attention
  • class activation map
  • epistemic uncertainty
  • explanation
  • visual question answering

ASJC Scopus subject areas

  • Software
  • Computer Graphics and Computer-Aided Design

Fingerprint

Dive into the research topics of 'Uncertainty Class Activation Map (U-CAM) using Gradient Certainty method'. Together they form a unique fingerprint.

Cite this