Recognition of human activity and the state of an assembly task using vision and inertial sensor fusion methods

Research output: Chapter or section in a book/report/conference proceedingChapter in a published conference proceeding

8 Citations (SciVal)
82 Downloads (Pure)

Abstract

Reliable human machine interfaces is key to accomplishing the goals of Industry 4.0. This work proposes the late fusion of a visual recognition and human action recognition (HAR) classifier. Vision is used to recognise the number of screws assembled into a mock part while HAR from body worn Inertial Measurement Units (IMUs) classifies actions done to assemble the part. Convolutional Neural Network (CNN) methods are used in both modes of classification before various late fusion methods are analysed for prediction of a final state estimate. The fusion methods investigated are mean, weighted average, Support Vector Machine (SVM), Bayesian, Artificial Neural Network (ANN) and Long Short Term Memory (LSTM). The results show the LSTM fusion method to perform best, with accuracy of 93% compared to 81% for IMU and 77% for visual sensing. Development of sensor fusion methods such as these is key to reliable Human Machine Interaction (HMI)
Original languageEnglish
Title of host publicationIEEE International Conference on Industrial Technology
PublisherIEEE
Number of pages6
ISBN (Electronic)978-1-7281-5730-6
DOIs
Publication statusPublished - 18 Jun 2021

Fingerprint

Dive into the research topics of 'Recognition of human activity and the state of an assembly task using vision and inertial sensor fusion methods'. Together they form a unique fingerprint.

Cite this