RECURSIVE DISENTANGLEMENT NETWORK

Yixuan Chen; Yubin Shi; Dongsheng Li; Yujiang Wang; Mingzhi Dong; Yingying Zhao; Robert Dick; Qin Lv; Fan Yang; Li Shang

RECURSIVE DISENTANGLEMENT NETWORK

Yixuan Chen, Yubin Shi, Dongsheng Li, Yujiang Wang, Mingzhi Dong, Yingying Zhao, Robert Dick, Qin Lv, Fan Yang, Li Shang

Research output: Contribution to conference › Paper › peer-review

1 Citation (SciVal)

Abstract

Disentangled feature representation is essential for data-efficient learning. The feature space of deep models is inherently compositional. Existing β-VAE-based methods, which only apply disentanglement regularization to the resulting embedding space of deep models, cannot effectively regularize such compositional feature space, resulting in unsatisfactory disentangled results. In this paper, we formulate the compositional disentanglement learning problem from an information-theoretic perspective and propose a recursive disentanglement network (RecurD) that propagates regulatory inductive bias recursively across the compositional feature space during disentangled representation learning. Experimental studies demonstrate that RecurD outperforms β-VAE and several of its state-of-the-art variants on disentangled representation learning and enables more data-efficient downstream machine learning tasks.

Original language	English
Publication status	Published - 2022
Event	10th International Conference on Learning Representations, ICLR 2022 - Virtual, Online Duration: 25 Apr 2022 → 29 Apr 2022

Conference

Conference	10th International Conference on Learning Representations, ICLR 2022
City	Virtual, Online
Period	25/04/22 → 29/04/22

Bibliographical note

Publisher Copyright:
© 2022 ICLR 2022 - 10th International Conference on Learning Representationss. All rights reserved.

ASJC Scopus subject areas

Language and Linguistics
Computer Science Applications
Education
Linguistics and Language

Cite this

@conference{14c7e421273f41248409ce4120b7cb3d,

title = "RECURSIVE DISENTANGLEMENT NETWORK",

abstract = "Disentangled feature representation is essential for data-efficient learning. The feature space of deep models is inherently compositional. Existing β-VAE-based methods, which only apply disentanglement regularization to the resulting embedding space of deep models, cannot effectively regularize such compositional feature space, resulting in unsatisfactory disentangled results. In this paper, we formulate the compositional disentanglement learning problem from an information-theoretic perspective and propose a recursive disentanglement network (RecurD) that propagates regulatory inductive bias recursively across the compositional feature space during disentangled representation learning. Experimental studies demonstrate that RecurD outperforms β-VAE and several of its state-of-the-art variants on disentangled representation learning and enables more data-efficient downstream machine learning tasks.",

author = "Yixuan Chen and Yubin Shi and Dongsheng Li and Yujiang Wang and Mingzhi Dong and Yingying Zhao and Robert Dick and Qin Lv and Fan Yang and Li Shang",

note = "Publisher Copyright: {\textcopyright} 2022 ICLR 2022 - 10th International Conference on Learning Representationss. All rights reserved.; 10th International Conference on Learning Representations, ICLR 2022 ; Conference date: 25-04-2022 Through 29-04-2022",

year = "2022",

language = "English",

}

TY - CONF

T1 - RECURSIVE DISENTANGLEMENT NETWORK

AU - Chen, Yixuan

AU - Shi, Yubin

AU - Li, Dongsheng

AU - Wang, Yujiang

AU - Dong, Mingzhi

AU - Zhao, Yingying

AU - Dick, Robert

AU - Lv, Qin

AU - Yang, Fan

AU - Shang, Li

PY - 2022

Y1 - 2022

N2 - Disentangled feature representation is essential for data-efficient learning. The feature space of deep models is inherently compositional. Existing β-VAE-based methods, which only apply disentanglement regularization to the resulting embedding space of deep models, cannot effectively regularize such compositional feature space, resulting in unsatisfactory disentangled results. In this paper, we formulate the compositional disentanglement learning problem from an information-theoretic perspective and propose a recursive disentanglement network (RecurD) that propagates regulatory inductive bias recursively across the compositional feature space during disentangled representation learning. Experimental studies demonstrate that RecurD outperforms β-VAE and several of its state-of-the-art variants on disentangled representation learning and enables more data-efficient downstream machine learning tasks.

AB - Disentangled feature representation is essential for data-efficient learning. The feature space of deep models is inherently compositional. Existing β-VAE-based methods, which only apply disentanglement regularization to the resulting embedding space of deep models, cannot effectively regularize such compositional feature space, resulting in unsatisfactory disentangled results. In this paper, we formulate the compositional disentanglement learning problem from an information-theoretic perspective and propose a recursive disentanglement network (RecurD) that propagates regulatory inductive bias recursively across the compositional feature space during disentangled representation learning. Experimental studies demonstrate that RecurD outperforms β-VAE and several of its state-of-the-art variants on disentangled representation learning and enables more data-efficient downstream machine learning tasks.

UR - https://www.scopus.com/pages/publications/85150339467

M3 - Paper

T2 - 10th International Conference on Learning Representations, ICLR 2022

Y2 - 25 April 2022 through 29 April 2022

ER -

RECURSIVE DISENTANGLEMENT NETWORK

Abstract

Conference

Bibliographical note

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this