Hierarchical Subquery Evaluation for Active Learning on a Graph

O. Mac Aodha, Neill Campbell, Jan Kautz, G.J. Brostow

Research output: Chapter in Book/Report/Conference proceedingConference contribution

35 Citations (Scopus)
64 Downloads (Pure)

Abstract

To train good supervised and semi-supervised object classifiers, it is critical that we not waste the time of the human experts who are providing the training labels. Existing active learning strategies can have uneven performance, being efficient on some datasets but wasteful on others, or inconsistent just between runs on the same dataset. We propose perplexity based graph construction and a new hierarchical subquery evaluation algorithm to combat this variability, and to release the potential of Expected Error Reduction. Under some specific circumstances, Expected Error Reduction has been one of the strongest-performing informativeness criteria for active learning. Until now, it has also been prohibitively costly to compute for sizeable datasets. We demonstrate our highly practical algorithm, comparing it to other active learning measures on classification datasets that vary in sparsity, dimensionality, and size. Our algorithm is consistent over multiple runs and achieves high accuracy, while querying the human expert for labels at a frequency that matches their desired time budget.
Original languageEnglish
Title of host publicationCVPR '14: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition
Place of PublicationSilver Spring, MD
PublisherIEEE
Pages564-571
Number of pages8
ISBN (Electronic)9781479951185
DOIs
Publication statusPublished - 2014
EventInternational Conference on Computer Vision and Pattern Recognition (CVPR) - Columbus, USA United States
Duration: 24 Jun 201427 Jun 2014

Publication series

NameIEEE Conference on Computer Vision and Pattern Recognition
ISSN (Electronic)1063-6919

Conference

ConferenceInternational Conference on Computer Vision and Pattern Recognition (CVPR)
CountryUSA United States
CityColumbus
Period24/06/1427/06/14

Cite this

Mac Aodha, O., Campbell, N., Kautz, J., & Brostow, G. J. (2014). Hierarchical Subquery Evaluation for Active Learning on a Graph. In CVPR '14: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (pp. 564-571). (IEEE Conference on Computer Vision and Pattern Recognition). Silver Spring, MD: IEEE. https://doi.org/10.1109/CVPR.2014.79

Hierarchical Subquery Evaluation for Active Learning on a Graph. / Mac Aodha, O.; Campbell, Neill; Kautz, Jan; Brostow, G.J.

CVPR '14: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Silver Spring, MD : IEEE, 2014. p. 564-571 (IEEE Conference on Computer Vision and Pattern Recognition).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Mac Aodha, O, Campbell, N, Kautz, J & Brostow, GJ 2014, Hierarchical Subquery Evaluation for Active Learning on a Graph. in CVPR '14: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Silver Spring, MD, pp. 564-571, International Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, USA United States, 24/06/14. https://doi.org/10.1109/CVPR.2014.79
Mac Aodha O, Campbell N, Kautz J, Brostow GJ. Hierarchical Subquery Evaluation for Active Learning on a Graph. In CVPR '14: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Silver Spring, MD: IEEE. 2014. p. 564-571. (IEEE Conference on Computer Vision and Pattern Recognition). https://doi.org/10.1109/CVPR.2014.79
Mac Aodha, O. ; Campbell, Neill ; Kautz, Jan ; Brostow, G.J. / Hierarchical Subquery Evaluation for Active Learning on a Graph. CVPR '14: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Silver Spring, MD : IEEE, 2014. pp. 564-571 (IEEE Conference on Computer Vision and Pattern Recognition).
@inproceedings{e455c0961923464eb9f0aea248555da1,
title = "Hierarchical Subquery Evaluation for Active Learning on a Graph",
abstract = "To train good supervised and semi-supervised object classifiers, it is critical that we not waste the time of the human experts who are providing the training labels. Existing active learning strategies can have uneven performance, being efficient on some datasets but wasteful on others, or inconsistent just between runs on the same dataset. We propose perplexity based graph construction and a new hierarchical subquery evaluation algorithm to combat this variability, and to release the potential of Expected Error Reduction. Under some specific circumstances, Expected Error Reduction has been one of the strongest-performing informativeness criteria for active learning. Until now, it has also been prohibitively costly to compute for sizeable datasets. We demonstrate our highly practical algorithm, comparing it to other active learning measures on classification datasets that vary in sparsity, dimensionality, and size. Our algorithm is consistent over multiple runs and achieves high accuracy, while querying the human expert for labels at a frequency that matches their desired time budget.",
author = "{Mac Aodha}, O. and Neill Campbell and Jan Kautz and G.J. Brostow",
year = "2014",
doi = "10.1109/CVPR.2014.79",
language = "English",
series = "IEEE Conference on Computer Vision and Pattern Recognition",
publisher = "IEEE",
pages = "564--571",
booktitle = "CVPR '14: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition",
address = "USA United States",

}

TY - GEN

T1 - Hierarchical Subquery Evaluation for Active Learning on a Graph

AU - Mac Aodha, O.

AU - Campbell, Neill

AU - Kautz, Jan

AU - Brostow, G.J.

PY - 2014

Y1 - 2014

N2 - To train good supervised and semi-supervised object classifiers, it is critical that we not waste the time of the human experts who are providing the training labels. Existing active learning strategies can have uneven performance, being efficient on some datasets but wasteful on others, or inconsistent just between runs on the same dataset. We propose perplexity based graph construction and a new hierarchical subquery evaluation algorithm to combat this variability, and to release the potential of Expected Error Reduction. Under some specific circumstances, Expected Error Reduction has been one of the strongest-performing informativeness criteria for active learning. Until now, it has also been prohibitively costly to compute for sizeable datasets. We demonstrate our highly practical algorithm, comparing it to other active learning measures on classification datasets that vary in sparsity, dimensionality, and size. Our algorithm is consistent over multiple runs and achieves high accuracy, while querying the human expert for labels at a frequency that matches their desired time budget.

AB - To train good supervised and semi-supervised object classifiers, it is critical that we not waste the time of the human experts who are providing the training labels. Existing active learning strategies can have uneven performance, being efficient on some datasets but wasteful on others, or inconsistent just between runs on the same dataset. We propose perplexity based graph construction and a new hierarchical subquery evaluation algorithm to combat this variability, and to release the potential of Expected Error Reduction. Under some specific circumstances, Expected Error Reduction has been one of the strongest-performing informativeness criteria for active learning. Until now, it has also been prohibitively costly to compute for sizeable datasets. We demonstrate our highly practical algorithm, comparing it to other active learning measures on classification datasets that vary in sparsity, dimensionality, and size. Our algorithm is consistent over multiple runs and achieves high accuracy, while querying the human expert for labels at a frequency that matches their desired time budget.

U2 - 10.1109/CVPR.2014.79

DO - 10.1109/CVPR.2014.79

M3 - Conference contribution

T3 - IEEE Conference on Computer Vision and Pattern Recognition

SP - 564

EP - 571

BT - CVPR '14: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition

PB - IEEE

CY - Silver Spring, MD

ER -