Efficient model selection for regularized classification by exploiting unlabeled data

Georgios Balikas; Ioannis Partalas; Eric Gaussier; Rohit Babbar; Massih Reza Amini

doi:10.1007/978-3-319-24465-5_3

Efficient model selection for regularized classification by exploiting unlabeled data

Georgios Balikas, Ioannis Partalas, Eric Gaussier, Rohit Babbar, Massih Reza Amini

Department of Computer Science

Research output: Chapter or section in a book/report/conference proceeding › Chapter in a published conference proceeding

4 Citations (SciVal)

Abstract

Hyper-parameter tuning is a resource-intensive task when optimizing classification models. The commonly used k-fold cross validation can become intractable in large scale settings when a classifier has to learn billions of parameters. At the same time, in real-world, one often encounters multi-class classification scenarios with only a few labeled examples; model selection approaches often offer little improvement in such cases and the default values of learners are used.We propose bounds for classification on accuracy and macro measures (precision, recall, F1) that motivate efficient schemes for model selection and can benefit from the existence of unlabeled data. We demonstrate the advantages of those schemes by comparing them with k-fold cross validation and hold-out estimation in the setting of large scale classification.

Original language	English
Title of host publication	Advances in Intelligent Data Analysis XIV - 14th International Symposium, IDA 2015, Proceedings
Editors	Tijl De Bie, Matthijs van Leeuwen, Elisa Fromont
Publisher	Springer Verlag
Pages	25-36
Number of pages	12
ISBN (Electronic)	9783319244655
ISBN (Print)	9783319244648
DOIs	https://doi.org/10.1007/978-3-319-24465-5_3
Publication status	Published - 22 Nov 2015
Event	14th International Symposium on Intelligent Data Analysis, IDA 2015 - Saint Etienne, France Duration: 22 Oct 2015 → 24 Oct 2015

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	9385
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	14th International Symposium on Intelligent Data Analysis, IDA 2015
Country/Territory	France
City	Saint Etienne
Period	22/10/15 → 24/10/15

Bibliographical note

Funding Information:
This work is partially supported by the CIFRE N 28/2015 and by the LabEx PERSYVAL Lab ANR-11-LABX-0025.

Publisher Copyright:
© Springer International Publishing Switzerland 2015.

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-319-24465-5_3

Cite this

Balikas, G., Partalas, I., Gaussier, E., Babbar, R., & Amini, M. R. (2015). Efficient model selection for regularized classification by exploiting unlabeled data. In T. De Bie, M. van Leeuwen, & E. Fromont (Eds.), Advances in Intelligent Data Analysis XIV - 14th International Symposium, IDA 2015, Proceedings (pp. 25-36). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9385). Springer Verlag. https://doi.org/10.1007/978-3-319-24465-5_3

Efficient model selection for regularized classification by exploiting unlabeled data. / Balikas, Georgios; Partalas, Ioannis; Gaussier, Eric et al.
Advances in Intelligent Data Analysis XIV - 14th International Symposium, IDA 2015, Proceedings. ed. / Tijl De Bie; Matthijs van Leeuwen; Elisa Fromont. Springer Verlag, 2015. p. 25-36 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9385).

Research output: Chapter or section in a book/report/conference proceeding › Chapter in a published conference proceeding

Balikas, G, Partalas, I, Gaussier, E, Babbar, R & Amini, MR 2015, Efficient model selection for regularized classification by exploiting unlabeled data. in T De Bie, M van Leeuwen & E Fromont (eds), Advances in Intelligent Data Analysis XIV - 14th International Symposium, IDA 2015, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9385, Springer Verlag, pp. 25-36, 14th International Symposium on Intelligent Data Analysis, IDA 2015, Saint Etienne, France, 22/10/15. https://doi.org/10.1007/978-3-319-24465-5_3

Balikas G, Partalas I, Gaussier E, Babbar R, Amini MR. Efficient model selection for regularized classification by exploiting unlabeled data. In De Bie T, van Leeuwen M, Fromont E, editors, Advances in Intelligent Data Analysis XIV - 14th International Symposium, IDA 2015, Proceedings. Springer Verlag. 2015. p. 25-36. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). Epub 2015 Nov 22. doi: 10.1007/978-3-319-24465-5_3

Balikas, Georgios ; Partalas, Ioannis ; Gaussier, Eric et al. / Efficient model selection for regularized classification by exploiting unlabeled data. Advances in Intelligent Data Analysis XIV - 14th International Symposium, IDA 2015, Proceedings. editor / Tijl De Bie ; Matthijs van Leeuwen ; Elisa Fromont. Springer Verlag, 2015. pp. 25-36 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{6ba05f93dea84d0cb89570fd89e67bf6,

title = "Efficient model selection for regularized classification by exploiting unlabeled data",

abstract = "Hyper-parameter tuning is a resource-intensive task when optimizing classification models. The commonly used k-fold cross validation can become intractable in large scale settings when a classifier has to learn billions of parameters. At the same time, in real-world, one often encounters multi-class classification scenarios with only a few labeled examples; model selection approaches often offer little improvement in such cases and the default values of learners are used.We propose bounds for classification on accuracy and macro measures (precision, recall, F1) that motivate efficient schemes for model selection and can benefit from the existence of unlabeled data. We demonstrate the advantages of those schemes by comparing them with k-fold cross validation and hold-out estimation in the setting of large scale classification.",

author = "Georgios Balikas and Ioannis Partalas and Eric Gaussier and Rohit Babbar and Amini, {Massih Reza}",

note = "Funding Information: This work is partially supported by the CIFRE N 28/2015 and by the LabEx PERSYVAL Lab ANR-11-LABX-0025. Publisher Copyright: {\textcopyright} Springer International Publishing Switzerland 2015.; 14th International Symposium on Intelligent Data Analysis, IDA 2015 ; Conference date: 22-10-2015 Through 24-10-2015",

year = "2015",

month = nov,

day = "22",

doi = "10.1007/978-3-319-24465-5_3",

language = "English",

isbn = "9783319244648",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "25--36",

editor = "{De Bie}, Tijl and {van Leeuwen}, Matthijs and Elisa Fromont",

booktitle = "Advances in Intelligent Data Analysis XIV - 14th International Symposium, IDA 2015, Proceedings",

}

TY - GEN

T1 - Efficient model selection for regularized classification by exploiting unlabeled data

AU - Balikas, Georgios

AU - Partalas, Ioannis

AU - Gaussier, Eric

AU - Babbar, Rohit

AU - Amini, Massih Reza

N1 - Funding Information: This work is partially supported by the CIFRE N 28/2015 and by the LabEx PERSYVAL Lab ANR-11-LABX-0025. Publisher Copyright: © Springer International Publishing Switzerland 2015.

PY - 2015/11/22

Y1 - 2015/11/22

N2 - Hyper-parameter tuning is a resource-intensive task when optimizing classification models. The commonly used k-fold cross validation can become intractable in large scale settings when a classifier has to learn billions of parameters. At the same time, in real-world, one often encounters multi-class classification scenarios with only a few labeled examples; model selection approaches often offer little improvement in such cases and the default values of learners are used.We propose bounds for classification on accuracy and macro measures (precision, recall, F1) that motivate efficient schemes for model selection and can benefit from the existence of unlabeled data. We demonstrate the advantages of those schemes by comparing them with k-fold cross validation and hold-out estimation in the setting of large scale classification.

AB - Hyper-parameter tuning is a resource-intensive task when optimizing classification models. The commonly used k-fold cross validation can become intractable in large scale settings when a classifier has to learn billions of parameters. At the same time, in real-world, one often encounters multi-class classification scenarios with only a few labeled examples; model selection approaches often offer little improvement in such cases and the default values of learners are used.We propose bounds for classification on accuracy and macro measures (precision, recall, F1) that motivate efficient schemes for model selection and can benefit from the existence of unlabeled data. We demonstrate the advantages of those schemes by comparing them with k-fold cross validation and hold-out estimation in the setting of large scale classification.

UR - http://www.scopus.com/inward/record.url?scp=84952045477&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-24465-5_3

DO - 10.1007/978-3-319-24465-5_3

M3 - Chapter in a published conference proceeding

AN - SCOPUS:84952045477

SN - 9783319244648

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 25

EP - 36

BT - Advances in Intelligent Data Analysis XIV - 14th International Symposium, IDA 2015, Proceedings

A2 - De Bie, Tijl

A2 - van Leeuwen, Matthijs

A2 - Fromont, Elisa

PB - Springer Verlag

T2 - 14th International Symposium on Intelligent Data Analysis, IDA 2015

Y2 - 22 October 2015 through 24 October 2015

ER -

Efficient model selection for regularized classification by exploiting unlabeled data

Abstract

Publication series

Conference

Bibliographical note

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this