Continual Learning Improves Zero-Shot Action Recognition

Shreyank N. Gowda, Davide Moltisanti, Laura Sevilla-Lara

Research output: Chapter or section in a book/report/conference proceedingChapter in a published conference proceeding

Abstract

Zero-shot action recognition requires a strong ability to generalize from pre-training and seen classes to novel unseen classes. Similarly, continual learning aims to develop models that can generalize effectively and learn new tasks without forgetting the ones previously learned. The generalization goals of zero-shot and continual learning are closely aligned, however techniques from continual learning have not been applied to zero-shot action recognition. In this paper, we propose a novel method based on continual learning to address zero-shot action recognition. This model, which we call Generative Iterative Learning (GIL) uses a memory of synthesized features of past classes, and combines these synthetic features with real ones from novel classes. The memory is used to train a classification model, ensuring a balanced exposure to both old and new classes. Experiments demonstrate that GIL improves generalization in unseen classes, achieving a new state-of-the-art in zero-shot recognition across multiple benchmarks. Importantly, GIL also boosts performance in the more challenging generalized zero-shot setting, where models need to retain knowledge about classes seen before fine-tuning.

Original languageEnglish
Title of host publicationComputer Vision – ACCV 2024 - 17th Asian Conference on Computer Vision, Proceedings
EditorsMinsu Cho, Ivan Laptev, Du Tran, Angela Yao, Hongbin Zha
Place of PublicationSingapore
PublisherSpringer, Singapore
Pages403-421
Number of pages19
ISBN (Electronic)9789819609086
ISBN (Print)9789819609079
DOIs
Publication statusPublished - 7 Dec 2024
Event17th Asian Conference on Computer Vision, ACCV 2024 - Hanoi, Viet Nam
Duration: 8 Dec 202412 Dec 2024

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume15474 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference17th Asian Conference on Computer Vision, ACCV 2024
Country/TerritoryViet Nam
CityHanoi
Period8/12/2412/12/24

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science

Fingerprint

Dive into the research topics of 'Continual Learning Improves Zero-Shot Action Recognition'. Together they form a unique fingerprint.

Cite this