Auto QA: The Question Is Not only What, but Also Where

Sumit Kumar, Badri N. Patro, Vinay P. Namboodiri

Research output: Chapter or section in a book/report/conference proceedingChapter in a published conference proceeding

1 Downloads (Pure)

Abstract

Visual Question Answering can be a functionally relevant task if purposed as such. In this paper, we aim to investigate and evaluate its efficacy in terms of localization-based question answering. We do this specifically in the context of autonomous driving where this functionality is important. To achieve our aim, we provide a new dataset, Auto-QA. Our new dataset is built over the Argoverse dataset and provides a truly multi-modal setting with seven views per frame and point-cloud LIDAR data being available for answering a localization-based question. We contribute localized attention adaptations of most popular VQA baselines and evaluate them on this task. We also provide joint point-cloud and image-based baselines that perform well on this task. An additional evaluation that we perform is to analyse whether the attention module is accurate or not for the image-based VQA baselines. To summarize, through this work we thoroughly analyze the localization abilities through visual question answering for autonomous driving and provide a new benchmark task for the same. Our best joint baseline model achieves a useful 74.8% accuracy on this task. We release our dataset and source code for our baseline modules in the following webpage: https: //delta-lab-iitk.github.io/AUTO-QA/

Original languageEnglish
Title of host publication2022 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW)
Place of PublicationU. S. A.
PublisherIEEE
Pages272-281
Number of pages10
Volume2022
ISBN (Electronic)9781665458245
DOIs
Publication statusPublished - 15 Feb 2022
Event2022 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops, WACVW 2022 - Waikoloa, USA United States
Duration: 4 Jan 20228 Jan 2022

Publication series

NameProceedings - 2022 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops, WACVW 2022
ISSN (Electronic)2690-621X

Conference

Conference2022 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops, WACVW 2022
Country/TerritoryUSA United States
CityWaikoloa
Period4/01/228/01/22

ASJC Scopus subject areas

  • Computer Science Applications
  • Computer Vision and Pattern Recognition

Cite this