Real-Time online action detection forests using spatio-Temporal contexts

Seungryul Baek, Kwang In Kim, Tae Kyun Kim

Research output: Chapter in Book/Report/Conference proceedingConference contribution

  • 3 Citations

Abstract

Online action detection (OAD) is challenging since 1) robust yet computationally expensive features cannot be straightforwardly used due to the real-Time processing requirements and 2) the localization and classification of actions have to be performed even before they are fully observed. We propose a new random forest (RF)-based online action detection framework that addresses these challenges. Our algorithm uses computationally efficient skeletal joint features. High accuracy is achieved by using robust convolutional neural network (CNN)-based features which are extracted from the raw RGBD images, plus the temporal relationships between the current frame of interest, and the past and futures frames. While these high-quality features are not available in real-Time testing scenario, we demonstrate that they can be effectively exploited in training RF classifiers: We use these spatio-Temporal contexts to craft RF's new split functions improving RFs' leaf node statistics. Experiments with challenging MSRAction3D, G3D, and OAD datasets demonstrate that our algorithm significantly improves the accuracy over the state-of-The-Art on-line action detection algorithms while achieving the real-Time efficiency of existing skeleton-based RF classifiers.

LanguageEnglish
Title of host publicationProceedings - 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), 2017
PublisherIEEE
Pages158-167
Number of pages10
ISBN (Electronic)9781509048229
DOIs
StatusPublished - 11 May 2017
Event17th IEEE Winter Conference on Applications of Computer Vision, WACV 2017 - Santa Rosa, USA United States
Duration: 24 Mar 201731 Mar 2017

Conference

Conference17th IEEE Winter Conference on Applications of Computer Vision, WACV 2017
CountryUSA United States
CitySanta Rosa
Period24/03/1731/03/17

Fingerprint

Classifiers
Statistics
Neural networks
Testing
Processing
Experiments

ASJC Scopus subject areas

  • Computer Science Applications
  • Computer Vision and Pattern Recognition

Cite this

Baek, S., Kim, K. I., & Kim, T. K. (2017). Real-Time online action detection forests using spatio-Temporal contexts. In Proceedings - 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), 2017 (pp. 158-167). [7926608] IEEE. https://doi.org/10.1109/WACV.2017.25

Real-Time online action detection forests using spatio-Temporal contexts. / Baek, Seungryul; Kim, Kwang In; Kim, Tae Kyun.

Proceedings - 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), 2017. IEEE, 2017. p. 158-167 7926608.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Baek, S, Kim, KI & Kim, TK 2017, Real-Time online action detection forests using spatio-Temporal contexts. in Proceedings - 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), 2017., 7926608, IEEE, pp. 158-167, 17th IEEE Winter Conference on Applications of Computer Vision, WACV 2017, Santa Rosa, USA United States, 24/03/17. https://doi.org/10.1109/WACV.2017.25
Baek S, Kim KI, Kim TK. Real-Time online action detection forests using spatio-Temporal contexts. In Proceedings - 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), 2017. IEEE. 2017. p. 158-167. 7926608 https://doi.org/10.1109/WACV.2017.25
Baek, Seungryul ; Kim, Kwang In ; Kim, Tae Kyun. / Real-Time online action detection forests using spatio-Temporal contexts. Proceedings - 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), 2017. IEEE, 2017. pp. 158-167
@inproceedings{bca47777122147de87ca7ec0f43b3011,
title = "Real-Time online action detection forests using spatio-Temporal contexts",
abstract = "Online action detection (OAD) is challenging since 1) robust yet computationally expensive features cannot be straightforwardly used due to the real-Time processing requirements and 2) the localization and classification of actions have to be performed even before they are fully observed. We propose a new random forest (RF)-based online action detection framework that addresses these challenges. Our algorithm uses computationally efficient skeletal joint features. High accuracy is achieved by using robust convolutional neural network (CNN)-based features which are extracted from the raw RGBD images, plus the temporal relationships between the current frame of interest, and the past and futures frames. While these high-quality features are not available in real-Time testing scenario, we demonstrate that they can be effectively exploited in training RF classifiers: We use these spatio-Temporal contexts to craft RF's new split functions improving RFs' leaf node statistics. Experiments with challenging MSRAction3D, G3D, and OAD datasets demonstrate that our algorithm significantly improves the accuracy over the state-of-The-Art on-line action detection algorithms while achieving the real-Time efficiency of existing skeleton-based RF classifiers.",
author = "Seungryul Baek and Kim, {Kwang In} and Kim, {Tae Kyun}",
year = "2017",
month = "5",
day = "11",
doi = "10.1109/WACV.2017.25",
language = "English",
pages = "158--167",
booktitle = "Proceedings - 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), 2017",
publisher = "IEEE",
address = "USA United States",

}

TY - GEN

T1 - Real-Time online action detection forests using spatio-Temporal contexts

AU - Baek, Seungryul

AU - Kim, Kwang In

AU - Kim, Tae Kyun

PY - 2017/5/11

Y1 - 2017/5/11

N2 - Online action detection (OAD) is challenging since 1) robust yet computationally expensive features cannot be straightforwardly used due to the real-Time processing requirements and 2) the localization and classification of actions have to be performed even before they are fully observed. We propose a new random forest (RF)-based online action detection framework that addresses these challenges. Our algorithm uses computationally efficient skeletal joint features. High accuracy is achieved by using robust convolutional neural network (CNN)-based features which are extracted from the raw RGBD images, plus the temporal relationships between the current frame of interest, and the past and futures frames. While these high-quality features are not available in real-Time testing scenario, we demonstrate that they can be effectively exploited in training RF classifiers: We use these spatio-Temporal contexts to craft RF's new split functions improving RFs' leaf node statistics. Experiments with challenging MSRAction3D, G3D, and OAD datasets demonstrate that our algorithm significantly improves the accuracy over the state-of-The-Art on-line action detection algorithms while achieving the real-Time efficiency of existing skeleton-based RF classifiers.

AB - Online action detection (OAD) is challenging since 1) robust yet computationally expensive features cannot be straightforwardly used due to the real-Time processing requirements and 2) the localization and classification of actions have to be performed even before they are fully observed. We propose a new random forest (RF)-based online action detection framework that addresses these challenges. Our algorithm uses computationally efficient skeletal joint features. High accuracy is achieved by using robust convolutional neural network (CNN)-based features which are extracted from the raw RGBD images, plus the temporal relationships between the current frame of interest, and the past and futures frames. While these high-quality features are not available in real-Time testing scenario, we demonstrate that they can be effectively exploited in training RF classifiers: We use these spatio-Temporal contexts to craft RF's new split functions improving RFs' leaf node statistics. Experiments with challenging MSRAction3D, G3D, and OAD datasets demonstrate that our algorithm significantly improves the accuracy over the state-of-The-Art on-line action detection algorithms while achieving the real-Time efficiency of existing skeleton-based RF classifiers.

UR - http://www.scopus.com/inward/record.url?scp=85020190906&partnerID=8YFLogxK

UR - http://dx.doi.org/10.1109/WACV.2017.25

U2 - 10.1109/WACV.2017.25

DO - 10.1109/WACV.2017.25

M3 - Conference contribution

SP - 158

EP - 167

BT - Proceedings - 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), 2017

PB - IEEE

ER -