Towards Memory-Efficient Training for Extremely Large Output Spaces – Learning with 670k Labels on a Single Commodity GPU

Erik Schultheis; Rohit Babbar

doi:10.1007/978-3-031-43418-1_41

Towards Memory-Efficient Training for Extremely Large Output Spaces – Learning with 670k Labels on a Single Commodity GPU

Aalto University

Research output: Chapter or section in a book/report/conference proceeding › Chapter in a published conference proceeding

Abstract

In classification problems with large output spaces (up to millions of labels), the last layer can require an enormous amount of memory. Using sparse connectivity would drastically reduce the memory requirements, but as we show below, applied naïvely it can result in much diminished predictive performance. Fortunately, we found that this can be mitigated by introducing an intermediate layer of intermediate size. We further demonstrate that one can constrain the connectivity of the sparse layer to be of constant fan-in, in the sense that each output neuron will have the exact same number of incoming connections, which allows for more efficient implementations, especially on GPU hardware. The CUDA implementation of our approach is provided at https://github.com/xmc-aalto/ecml23-sparse.

Original language	English
Title of host publication	Machine Learning and Knowledge Discovery in Databases
Subtitle of host publication	Research Track - European Conference, ECML PKDD 2023, Proceedings
Editors	Danai Koutra, Claudia Plant, Manuel Gomez Rodriguez, Elena Baralis, Francesco Bonchi
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	689-704
Number of pages	16
ISBN (Print)	9783031434174
DOIs	https://doi.org/10.1007/978-3-031-43418-1_41
Publication status	Published - 17 Sept 2023
Event	European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2023 - Turin, Italy Duration: 18 Sept 2023 → 22 Sept 2023

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	14171 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2023
Country/Territory	Italy
City	Turin
Period	18/09/23 → 22/09/23

Funding

We acknowledge the support of computational resources provided by the Aalto Science-IT project, and CSC IT Center for Science, Finland. This work is funded in part by the Academy of Finland projects 347707 and 348215.

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-031-43418-1_41

Cite this

Schultheis, E., & Babbar, R. (2023). Towards Memory-Efficient Training for Extremely Large Output Spaces – Learning with 670k Labels on a Single Commodity GPU. In D. Koutra, C. Plant, M. Gomez Rodriguez, E. Baralis, & F. Bonchi (Eds.), Machine Learning and Knowledge Discovery in Databases: Research Track - European Conference, ECML PKDD 2023, Proceedings (pp. 689-704). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14171 LNAI). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-43418-1_41

Towards Memory-Efficient Training for Extremely Large Output Spaces – Learning with 670k Labels on a Single Commodity GPU. / Schultheis, Erik; Babbar, Rohit.
Machine Learning and Knowledge Discovery in Databases: Research Track - European Conference, ECML PKDD 2023, Proceedings. ed. / Danai Koutra; Claudia Plant; Manuel Gomez Rodriguez; Elena Baralis; Francesco Bonchi. Springer Science and Business Media Deutschland GmbH, 2023. p. 689-704 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14171 LNAI).

Research output: Chapter or section in a book/report/conference proceeding › Chapter in a published conference proceeding

Schultheis, E & Babbar, R 2023, Towards Memory-Efficient Training for Extremely Large Output Spaces – Learning with 670k Labels on a Single Commodity GPU. in D Koutra, C Plant, M Gomez Rodriguez, E Baralis & F Bonchi (eds), Machine Learning and Knowledge Discovery in Databases: Research Track - European Conference, ECML PKDD 2023, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14171 LNAI, Springer Science and Business Media Deutschland GmbH, pp. 689-704, European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2023, Turin, Italy, 18/09/23. https://doi.org/10.1007/978-3-031-43418-1_41

Schultheis E, Babbar R. Towards Memory-Efficient Training for Extremely Large Output Spaces – Learning with 670k Labels on a Single Commodity GPU. In Koutra D, Plant C, Gomez Rodriguez M, Baralis E, Bonchi F, editors, Machine Learning and Knowledge Discovery in Databases: Research Track - European Conference, ECML PKDD 2023, Proceedings. Springer Science and Business Media Deutschland GmbH. 2023. p. 689-704. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). Epub 2023 Sept 17. doi: 10.1007/978-3-031-43418-1_41

Schultheis, Erik ; Babbar, Rohit. / Towards Memory-Efficient Training for Extremely Large Output Spaces – Learning with 670k Labels on a Single Commodity GPU. Machine Learning and Knowledge Discovery in Databases: Research Track - European Conference, ECML PKDD 2023, Proceedings. editor / Danai Koutra ; Claudia Plant ; Manuel Gomez Rodriguez ; Elena Baralis ; Francesco Bonchi. Springer Science and Business Media Deutschland GmbH, 2023. pp. 689-704 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{eaf633353ff2450bad9d21e133b3f98f,

title = "Towards Memory-Efficient Training for Extremely Large Output Spaces – Learning with 670k Labels on a Single Commodity GPU",

abstract = "In classification problems with large output spaces (up to millions of labels), the last layer can require an enormous amount of memory. Using sparse connectivity would drastically reduce the memory requirements, but as we show below, applied na{\"i}vely it can result in much diminished predictive performance. Fortunately, we found that this can be mitigated by introducing an intermediate layer of intermediate size. We further demonstrate that one can constrain the connectivity of the sparse layer to be of constant fan-in, in the sense that each output neuron will have the exact same number of incoming connections, which allows for more efficient implementations, especially on GPU hardware. The CUDA implementation of our approach is provided at https://github.com/xmc-aalto/ecml23-sparse.",

author = "Erik Schultheis and Rohit Babbar",

year = "2023",

month = sep,

day = "17",

doi = "10.1007/978-3-031-43418-1_41",

language = "English",

isbn = "9783031434174",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "689--704",

editor = "Danai Koutra and Claudia Plant and {Gomez Rodriguez}, Manuel and Elena Baralis and Francesco Bonchi",

booktitle = "Machine Learning and Knowledge Discovery in Databases",

address = "Germany",

note = "European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2023 ; Conference date: 18-09-2023 Through 22-09-2023",

}

TY - GEN

T1 - Towards Memory-Efficient Training for Extremely Large Output Spaces – Learning with 670k Labels on a Single Commodity GPU

AU - Schultheis, Erik

AU - Babbar, Rohit

PY - 2023/9/17

Y1 - 2023/9/17

N2 - In classification problems with large output spaces (up to millions of labels), the last layer can require an enormous amount of memory. Using sparse connectivity would drastically reduce the memory requirements, but as we show below, applied naïvely it can result in much diminished predictive performance. Fortunately, we found that this can be mitigated by introducing an intermediate layer of intermediate size. We further demonstrate that one can constrain the connectivity of the sparse layer to be of constant fan-in, in the sense that each output neuron will have the exact same number of incoming connections, which allows for more efficient implementations, especially on GPU hardware. The CUDA implementation of our approach is provided at https://github.com/xmc-aalto/ecml23-sparse.

AB - In classification problems with large output spaces (up to millions of labels), the last layer can require an enormous amount of memory. Using sparse connectivity would drastically reduce the memory requirements, but as we show below, applied naïvely it can result in much diminished predictive performance. Fortunately, we found that this can be mitigated by introducing an intermediate layer of intermediate size. We further demonstrate that one can constrain the connectivity of the sparse layer to be of constant fan-in, in the sense that each output neuron will have the exact same number of incoming connections, which allows for more efficient implementations, especially on GPU hardware. The CUDA implementation of our approach is provided at https://github.com/xmc-aalto/ecml23-sparse.

UR - http://www.scopus.com/inward/record.url?scp=85174438309&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-43418-1_41

DO - 10.1007/978-3-031-43418-1_41

M3 - Chapter in a published conference proceeding

AN - SCOPUS:85174438309

SN - 9783031434174

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 689

EP - 704

BT - Machine Learning and Knowledge Discovery in Databases

A2 - Koutra, Danai

A2 - Plant, Claudia

A2 - Gomez Rodriguez, Manuel

A2 - Baralis, Elena

A2 - Bonchi, Francesco

PB - Springer Science and Business Media Deutschland GmbH

T2 - European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2023

Y2 - 18 September 2023 through 22 September 2023

ER -