Recognition of human activity and the state of an assembly task using vision and inertial sensor fusion methods

James Male; Uriel Martinez Hernandez

doi:10.1109/ICIT46573.2021.9453672

Recognition of human activity and the state of an assembly task using vision and inertial sensor fusion methods

Research output: Chapter or section in a book/report/conference proceeding › Chapter in a published conference proceeding

11 Citations (SciVal)

155 Downloads (Pure)

Abstract

Reliable human machine interfaces is key to accomplishing the goals of Industry 4.0. This work proposes the late fusion of a visual recognition and human action recognition (HAR) classifier. Vision is used to recognise the number of screws assembled into a mock part while HAR from body worn Inertial Measurement Units (IMUs) classifies actions done to assemble the part. Convolutional Neural Network (CNN) methods are used in both modes of classification before various late fusion methods are analysed for prediction of a final state estimate. The fusion methods investigated are mean, weighted average, Support Vector Machine (SVM), Bayesian, Artificial Neural Network (ANN) and Long Short Term Memory (LSTM). The results show the LSTM fusion method to perform best, with accuracy of 93% compared to 81% for IMU and 77% for visual sensing. Development of sensor fusion methods such as these is key to reliable Human Machine Interaction (HMI)

Original language	English
Title of host publication	IEEE International Conference on Industrial Technology
Publisher	IEEE
Number of pages	6
ISBN (Electronic)	978-1-7281-5730-6
DOIs	https://doi.org/10.1109/ICIT46573.2021.9453672
Publication status	Published - 18 Jun 2021

Access to Document

10.1109/ICIT46573.2021.9453672

ICIT_Conference_Paper_accepted_version.PDF
© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.
Accepted author manuscript, 5.67 MB

Cite this

@inproceedings{52ca7124a4284a7eb195507556ce0ad5,

title = "Recognition of human activity and the state of an assembly task using vision and inertial sensor fusion methods",

abstract = "Reliable human machine interfaces is key to accomplishing the goals of Industry 4.0. This work proposes the late fusion of a visual recognition and human action recognition (HAR) classifier. Vision is used to recognise the number of screws assembled into a mock part while HAR from body worn Inertial Measurement Units (IMUs) classifies actions done to assemble the part. Convolutional Neural Network (CNN) methods are used in both modes of classification before various late fusion methods are analysed for prediction of a final state estimate. The fusion methods investigated are mean, weighted average, Support Vector Machine (SVM), Bayesian, Artificial Neural Network (ANN) and Long Short Term Memory (LSTM). The results show the LSTM fusion method to perform best, with accuracy of 93% compared to 81% for IMU and 77% for visual sensing. Development of sensor fusion methods such as these is key to reliable Human Machine Interaction (HMI)",

author = "James Male and {Martinez Hernandez}, Uriel",

year = "2021",

month = jun,

day = "18",

doi = "10.1109/ICIT46573.2021.9453672",

language = "English",

booktitle = "IEEE International Conference on Industrial Technology",

publisher = "IEEE",

address = "USA United States",

}

TY - GEN

T1 - Recognition of human activity and the state of an assembly task using vision and inertial sensor fusion methods

AU - Male, James

AU - Martinez Hernandez, Uriel

PY - 2021/6/18

Y1 - 2021/6/18

N2 - Reliable human machine interfaces is key to accomplishing the goals of Industry 4.0. This work proposes the late fusion of a visual recognition and human action recognition (HAR) classifier. Vision is used to recognise the number of screws assembled into a mock part while HAR from body worn Inertial Measurement Units (IMUs) classifies actions done to assemble the part. Convolutional Neural Network (CNN) methods are used in both modes of classification before various late fusion methods are analysed for prediction of a final state estimate. The fusion methods investigated are mean, weighted average, Support Vector Machine (SVM), Bayesian, Artificial Neural Network (ANN) and Long Short Term Memory (LSTM). The results show the LSTM fusion method to perform best, with accuracy of 93% compared to 81% for IMU and 77% for visual sensing. Development of sensor fusion methods such as these is key to reliable Human Machine Interaction (HMI)

AB - Reliable human machine interfaces is key to accomplishing the goals of Industry 4.0. This work proposes the late fusion of a visual recognition and human action recognition (HAR) classifier. Vision is used to recognise the number of screws assembled into a mock part while HAR from body worn Inertial Measurement Units (IMUs) classifies actions done to assemble the part. Convolutional Neural Network (CNN) methods are used in both modes of classification before various late fusion methods are analysed for prediction of a final state estimate. The fusion methods investigated are mean, weighted average, Support Vector Machine (SVM), Bayesian, Artificial Neural Network (ANN) and Long Short Term Memory (LSTM). The results show the LSTM fusion method to perform best, with accuracy of 93% compared to 81% for IMU and 77% for visual sensing. Development of sensor fusion methods such as these is key to reliable Human Machine Interaction (HMI)

U2 - 10.1109/ICIT46573.2021.9453672

DO - 10.1109/ICIT46573.2021.9453672

M3 - Chapter in a published conference proceeding

BT - IEEE International Conference on Industrial Technology

PB - IEEE

ER -

Recognition of human activity and the state of an assembly task using vision and inertial sensor fusion methods

Abstract

Access to Document

Fingerprint

Cite this